[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:21 dist-test-slave-rwrg krb5kdc[6348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:21 dist-test-slave-rwrg krb5kdc[6348](info): set up 2 sockets
Aug 10 22:58:21 dist-test-slave-rwrg krb5kdc[6348](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 22:58:23.007809 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:23.130971 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33061
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:33061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:23.225417 6369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:23.225649 6369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:23.225697 6369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:23.226874 6369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:23.226941 6369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:23.226967 6369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:23.226998 6369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:23.227021 6369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:23.228775 6369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:33061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:23.229060 6369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:23.229307 6369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:23.234465 6378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.234522 6377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.234663 6382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:23.234733 6369 server_base.cc:1034] running on GCE node
I20260810 22:58:23.234954 6369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:23.235203 6369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:23.236351 6369 hybrid_clock.cc:648] HybridClock initialized: now 1786402703236338 us; error 80 us; skew 500 ppm
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:23.239223 6369 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:23.239639 6369 webserver.cc:460] Webserver started at http://127.6.47.190:38713/ using document root <none> and password file <none>
I20260810 22:58:23.239884 6369 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:23.239955 6369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:23.240114 6369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:23.241580 6369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "573d5f7758614362b764a3eec9488a05"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "a8871d2b7311fcc17f7bacb108586b5e"
server_key_iv: "5994d11eca04c84c55eadc9f5d82ba8f"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.241977 6369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "573d5f7758614362b764a3eec9488a05"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "a8871d2b7311fcc17f7bacb108586b5e"
server_key_iv: "5994d11eca04c84c55eadc9f5d82ba8f"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.243396 6369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:23.244235 6391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.244401 6369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:23.244521 6369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "573d5f7758614362b764a3eec9488a05"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "a8871d2b7311fcc17f7bacb108586b5e"
server_key_iv: "5994d11eca04c84c55eadc9f5d82ba8f"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.244608 6369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:23.252930 6369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:23.254951 6369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:23.255118 6369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:23.259424 6369 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:33061
I20260810 22:58:23.259455 6481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:33061 every 8 connection(s)
I20260810 22:58:23.259796 6369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:23.261083 6483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:23.264241 6483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05: Bootstrap starting.
I20260810 22:58:23.264942 6483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:23.265249 6483 log.cc:824] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:23.266019 6483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05: No bootstrap required, opened a new log
I20260810 22:58:23.266897 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6369
I20260810 22:58:23.267009 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:23.267642 6334 external_mini_cluster.cc:1428] Setting key 82ad3701593bd6eb5551869b22724174
I20260810 22:58:23.267835 6483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } }
I20260810 22:58:23.267972 6483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:23.267997 6483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573d5f7758614362b764a3eec9488a05, State: Initialized, Role: FOLLOWER
I20260810 22:58:23.268189 6483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } }
I20260810 22:58:23.268275 6483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:23.268329 6483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:23.268381 6483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:23.269560 6483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } }
I20260810 22:58:23.269680 6483 leader_election.cc:302] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 573d5f7758614362b764a3eec9488a05; no voters:
I20260810 22:58:23.269840 6483 leader_election.cc:288] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:23.269870 6497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:23.270074 6497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [term 1 LEADER]: Becoming Leader. State: Replica: 573d5f7758614362b764a3eec9488a05, State: Running, Role: LEADER
I20260810 22:58:23.270092 6483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:23.270170 6497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } }
I20260810 22:58:23.270570 6499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "573d5f7758614362b764a3eec9488a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } } }
I20260810 22:58:23.270562 6497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d5f7758614362b764a3eec9488a05. Latest consensus state: current_term: 1 leader_uuid: "573d5f7758614362b764a3eec9488a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d5f7758614362b764a3eec9488a05" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33061 } } }
I20260810 22:58:23.270730 6499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:23.270736 6497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:23.271217 6517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:23.272051 6517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:23.273703 6517 catalog_manager.cc:1347] Generated new cluster ID: b4182fd5220f482a999e33d01773cb4d
I20260810 22:58:23.273758 6517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:23.283829 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.268565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57276 (local address 127.6.47.190:33061)
0810 22:58:23.268787 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.268796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.269090 (+ 294us) server_negotiation.cc:407] Connection header received
0810 22:58:23.271741 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.271750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.271843 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.271989 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.274193 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.275072 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.276084 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.276481 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.276655 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.280014 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.280025 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.280025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.280035 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.281778 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.282261 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.282264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.282265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.282367 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.282637 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.282638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.282639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.282891 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.283071 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.283343 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.283580 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
I20260810 22:58:23.296523 6517 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:23.297151 6517 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:23.303557 6517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573d5f7758614362b764a3eec9488a05: Generated new TSK 0
I20260810 22:58:23.303772 6517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:23.339555 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:23.445037 6534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:23.445421 6534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:23.445487 6534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:23.446625 6534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:23.446666 6534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:23.446723 6534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:23.448351 6534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:23.448632 6534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:23.448911 6534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:23.454382 6551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.454397 6547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.454484 6548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:23.454726 6534 server_base.cc:1034] running on GCE node
I20260810 22:58:23.454953 6534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:23.455205 6534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:23.456370 6534 hybrid_clock.cc:648] HybridClock initialized: now 1786402703456327 us; error 65 us; skew 500 ppm
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:23.458969 6534 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:23.459414 6534 webserver.cc:460] Webserver started at http://127.6.47.129:39485/ using document root <none> and password file <none>
I20260810 22:58:23.459662 6534 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:23.459743 6534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:23.459897 6534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:23.461015 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "91a54bcff7a24b27918a5593a4466afa"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "4dba1defe803b5edc653fb9feb1dbff1"
server_key_iv: "696a99bd7b94a1dc5c5fa24ceebcabde"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.461402 6534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "91a54bcff7a24b27918a5593a4466afa"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "4dba1defe803b5edc653fb9feb1dbff1"
server_key_iv: "696a99bd7b94a1dc5c5fa24ceebcabde"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.462821 6534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:23.463636 6564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.463831 6534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 22:58:23.463936 6534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "91a54bcff7a24b27918a5593a4466afa"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "4dba1defe803b5edc653fb9feb1dbff1"
server_key_iv: "696a99bd7b94a1dc5c5fa24ceebcabde"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.464010 6534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:23.478684 6534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:23.480485 6534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:23.480674 6534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:23.480907 6534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:23.481237 6534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:23.481293 6534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.481331 6534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:23.481357 6534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.488135 6534 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:35867
I20260810 22:58:23.488193 6771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:35867 every 8 connection(s)
I20260810 22:58:23.488547 6534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:23.496075 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6534
I20260810 22:58:23.496183 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:23.496352 6334 external_mini_cluster.cc:1428] Setting key 679037c5c2299fc7ec79d1b5c13795db
I20260810 22:58:23.498143 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.489403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:33061 (local address 127.6.47.129:45553)
0810 22:58:23.489576 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:23.489619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:23.490098 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:23.490309 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:23.490312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:23.490984 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:23.491229 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.491235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.491898 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.491899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.492727 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.492730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.493066 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.493067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.493164 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.493624 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:23.493633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:23.494948 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:23.496769 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.496777 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.496779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.497094 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.497318 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.497319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.497319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.497422 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.497695 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:23.497698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:23.497799 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 22:58:23.497873 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 22:58:23.497988 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260810 22:58:23.498154 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.489459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45553 (local address 127.6.47.190:33061)
0810 22:58:23.489510 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.489512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.489718 (+ 206us) server_negotiation.cc:407] Connection header received
0810 22:58:23.490189 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.490189 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.490198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.490256 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.491318 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.491814 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.492802 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.492989 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.493042 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.495038 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.495039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.495040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.495046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.496688 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.497184 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.497185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.497186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.497252 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.497488 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.497489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.497489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.497631 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.497709 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.497918 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.498013 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 22:58:23.498643 6773 heartbeater.cc:347] Connected to a master server at 127.6.47.190:33061
I20260810 22:58:23.498765 6773 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:23.498960 6773 heartbeater.cc:511] Master 127.6.47.190:33061 requested a full tablet report, sending...
I20260810 22:58:23.499565 6421 ts_manager.cc:194] Registered new tserver with Master: 91a54bcff7a24b27918a5593a4466afa (127.6.47.129:35867)
I20260810 22:58:23.500208 6421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:45553
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:23.525167 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:23.610389 6782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:23.610610 6782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:23.610633 6782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:23.611689 6782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:23.611722 6782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:23.611747 6782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:23.613153 6782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:23.613353 6782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:23.613540 6782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:23.618559 6789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.618595 6791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.618631 6795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:23.618709 6782 server_base.cc:1034] running on GCE node
I20260810 22:58:23.618942 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:23.619199 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:23.620364 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786402703620337 us; error 39 us; skew 500 ppm
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:23.622952 6782 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:23.623409 6782 webserver.cc:460] Webserver started at http://127.6.47.130:38631/ using document root <none> and password file <none>
I20260810 22:58:23.623601 6782 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:23.623643 6782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:23.623724 6782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:23.624771 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "abd68485b4ad4c4686e959d8a0216079"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "17708ac6a5d089d0a62f22a2fb752773"
server_key_iv: "7b51238b0cd5e6e647255fe8d7972d86"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.625095 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "abd68485b4ad4c4686e959d8a0216079"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "17708ac6a5d089d0a62f22a2fb752773"
server_key_iv: "7b51238b0cd5e6e647255fe8d7972d86"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.626340 6782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:23.627074 6804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.627300 6782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:23.627434 6782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "abd68485b4ad4c4686e959d8a0216079"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "17708ac6a5d089d0a62f22a2fb752773"
server_key_iv: "7b51238b0cd5e6e647255fe8d7972d86"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.627522 6782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:23.650287 6782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:23.653155 6782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:23.653354 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:23.653656 6782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:23.654036 6782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:23.654081 6782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.654145 6782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:23.654183 6782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.662943 6782 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:33045
I20260810 22:58:23.663000 7003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:33045 every 8 connection(s)
I20260810 22:58:23.663360 6782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:23.671262 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6782
I20260810 22:58:23.671365 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:23.671567 6334 external_mini_cluster.cc:1428] Setting key 3d5aa0ec8ffaa3fa8c050888d15f0d59
I20260810 22:58:23.673357 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.664272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:46551 (local address 127.6.47.190:33061)
0810 22:58:23.664367 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.664370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.664618 (+ 248us) server_negotiation.cc:407] Connection header received
0810 22:58:23.665178 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.665179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.665189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.665238 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.666257 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.666743 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.667804 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.668053 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.668106 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.670294 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.670295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.670295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.670301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.671837 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.672261 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.672263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.672263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.672327 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.672648 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.672650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.672650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.672793 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.672877 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.673163 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.673268 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 22:58:23.673511 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.664225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:33061 (local address 127.6.47.130:46551)
0810 22:58:23.664428 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:23.664468 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:23.665077 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:23.665307 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:23.665311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:23.665948 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:23.666173 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.666178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.666928 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.666929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.667724 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.667726 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.668134 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.668135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.668234 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.668807 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:23.668815 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:23.670195 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:23.671923 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.671931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.671933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.672161 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.672404 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.672405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.672405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.672569 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.672881 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:23.672889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:23.673026 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 22:58:23.673125 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 22:58:23.673283 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260810 22:58:23.673938 7005 heartbeater.cc:347] Connected to a master server at 127.6.47.190:33061
I20260810 22:58:23.674034 7005 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:23.674232 7005 heartbeater.cc:511] Master 127.6.47.190:33061 requested a full tablet report, sending...
I20260810 22:58:23.674599 6421 ts_manager.cc:194] Registered new tserver with Master: abd68485b4ad4c4686e959d8a0216079 (127.6.47.130:33045)
I20260810 22:58:23.675024 6421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:46551
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:23.699631 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:35601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:23.820699 7011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:23.821065 7011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:23.821117 7011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:23.823665 7011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:23.823763 7011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:23.823837 7011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:23.827576 7011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:33061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:23.827994 7011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:23.828434 7011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:23.839421 7019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:23.839520 7023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:23.839737 7011 server_base.cc:1034] running on GCE node
W20260810 22:58:23.839459 7018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:23.840013 7011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:23.840341 7011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:23.841511 7011 hybrid_clock.cc:648] HybridClock initialized: now 1786402703841502 us; error 69 us; skew 500 ppm
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:23.845373 7011 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:23.846060 7011 webserver.cc:460] Webserver started at http://127.6.47.131:33253/ using document root <none> and password file <none>
I20260810 22:58:23.846417 7011 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:23.846505 7011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:23.846689 7011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:23.848565 7011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "21a3cf937605404f807170446cf4db1b"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "927ce8456139fe99c99f5a4dab5fc18f"
server_key_iv: "e08a174c3443375e2aa4f05bd44392c0"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.849109 7011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "21a3cf937605404f807170446cf4db1b"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "927ce8456139fe99c99f5a4dab5fc18f"
server_key_iv: "e08a174c3443375e2aa4f05bd44392c0"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.851495 7011 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260810 22:58:23.852592 7035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.852797 7011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:23.852912 7011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "21a3cf937605404f807170446cf4db1b"
format_stamp: "Formatted at 2026-08-10 22:58:23 on dist-test-slave-rwrg"
server_key: "927ce8456139fe99c99f5a4dab5fc18f"
server_key_iv: "e08a174c3443375e2aa4f05bd44392c0"
server_key_version: "encryptionkey@0"
I20260810 22:58:23.853016 7011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:23.876364 7011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:23.878504 7011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:23.878669 7011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:23.879009 7011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:23.879364 7011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:23.879432 7011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.879492 7011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:23.879518 7011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:23.890612 7011 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:42261
I20260810 22:58:23.890666 7234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:42261 every 8 connection(s)
I20260810 22:58:23.891069 7011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:23.898554 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7011
I20260810 22:58:23.898645 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:23.898816 6334 external_mini_cluster.cc:1428] Setting key b856c26f4b13d4b3e3b570678175eba5
I20260810 22:58:23.900317 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.891831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:37567 (local address 127.6.47.190:33061)
0810 22:58:23.891907 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.891909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.892113 (+ 204us) server_negotiation.cc:407] Connection header received
0810 22:58:23.892672 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.892673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.892683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.892740 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.893623 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.894100 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.895031 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.895211 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.895271 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.897452 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.897454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.897454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.897461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.898797 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.899278 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.899279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.899280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.899341 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.899634 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.899635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.899635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.899770 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.899839 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.900187 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.900254 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:58:23.900475 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.891814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:33061 (local address 127.6.47.131:37567)
0810 22:58:23.891979 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:23.892023 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:23.892537 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:23.892796 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:23.892800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:23.893355 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:23.893543 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.893548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.894183 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.894184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.894955 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:23.894958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.895294 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.895295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:23.895389 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.895869 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:23.895877 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:23.897354 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:23.898860 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.898869 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.898871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.899179 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.899412 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:23.899413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:23.899413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:23.899556 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:23.899845 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:23.899852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:23.900038 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 22:58:23.900158 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 22:58:23.900281 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260810 22:58:23.900928 7235 heartbeater.cc:347] Connected to a master server at 127.6.47.190:33061
I20260810 22:58:23.901021 7235 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:23.901160 7235 heartbeater.cc:511] Master 127.6.47.190:33061 requested a full tablet report, sending...
I20260810 22:58:23.901515 6421 ts_manager.cc:194] Registered new tserver with Master: 21a3cf937605404f807170446cf4db1b (127.6.47.131:42261)
I20260810 22:58:23.901875 6421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:37567
I20260810 22:58:23.910799 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:23.936432 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.928320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.6.47.129:35867)
0810 22:58:23.928600 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.928605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.928616 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:23.928645 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.928648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.928662 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.928735 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.929312 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.930183 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.930983 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.931332 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.931391 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.933725 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.933735 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.933736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.933743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.935298 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.935613 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.935616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.935616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.935671 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.935917 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.935918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.935918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.936051 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.936165 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.936238 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.936302 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":130,"threads_started":1}
W20260810 22:58:23.936846 6615 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56246
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:58:23 dist-test-slave-rwrg krb5kdc[6348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:23.961607 7245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:23.954217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.6.47.129:35867)
0810 22:58:23.954309 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:23.954317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:23.954329 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:58:23.954393 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:23.954395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:23.954407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:23.954519 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:23.955083 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.955663 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.956377 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:23.956687 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:23.956755 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:23.959154 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:23.959155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:23.959156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:23.959174 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:23.960387 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.960765 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.960768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.960768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.960823 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:23.961031 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:23.961032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:23.961033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:23.961134 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:23.961371 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:23.961409 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 22:58:23.961493 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:58:23.962198 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6534
I20260810 22:58:23.978561 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6782
I20260810 22:58:24.005863 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7011
I20260810 22:58:24.014757 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6369
2026-08-10T22:58:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2317 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:24 dist-test-slave-rwrg krb5kdc[7258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:24 dist-test-slave-rwrg krb5kdc[7258](info): set up 2 sockets
Aug 10 22:58:24 dist-test-slave-rwrg krb5kdc[7258](info): commencing operation
krb5kdc: starting...
W20260810 22:58:25.294878 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:25.414952 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:45541
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:45541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:25.496659 7284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:25.496882 7284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:25.496920 7284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:25.498003 7284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:25.498068 7284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:25.498093 7284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:25.498116 7284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:25.498142 7284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:25.499622 7284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:45541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:45541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:25.499892 7284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:25.500113 7284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:25.505136 7293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:25.505219 7284 server_base.cc:1034] running on GCE node
W20260810 22:58:25.505157 7299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:25.505154 7297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:25.505587 7284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:25.505883 7284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:25.507077 7284 hybrid_clock.cc:648] HybridClock initialized: now 1786402705507059 us; error 30 us; skew 500 ppm
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:25.509563 7284 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:25.510046 7284 webserver.cc:460] Webserver started at http://127.6.47.190:39229/ using document root <none> and password file <none>
I20260810 22:58:25.510277 7284 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:25.510345 7284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:25.510502 7284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:25.511564 7284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "ceb10485cd8844269e590f8f59bc9ccc"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "0ac1f0e706ebda77bb20b27951d4db0f"
server_key_iv: "541b7b9eeb7b527b9c1e880dd2f0b822"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.511955 7284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "ceb10485cd8844269e590f8f59bc9ccc"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "0ac1f0e706ebda77bb20b27951d4db0f"
server_key_iv: "541b7b9eeb7b527b9c1e880dd2f0b822"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.513336 7284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 22:58:25.514071 7316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.514243 7284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:25.514350 7284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "ceb10485cd8844269e590f8f59bc9ccc"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "0ac1f0e706ebda77bb20b27951d4db0f"
server_key_iv: "541b7b9eeb7b527b9c1e880dd2f0b822"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.514425 7284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:25.532977 7284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:25.534957 7284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:25.535121 7284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:25.539427 7284 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:45541
I20260810 22:58:25.539469 7416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:45541 every 8 connection(s)
I20260810 22:58:25.539856 7284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:25.540295 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7284
I20260810 22:58:25.540400 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:25.540607 6334 external_mini_cluster.cc:1428] Setting key 20ebdacd2cc1f05d910a98537bfef125
I20260810 22:58:25.541047 7417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:25.543785 7417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc: Bootstrap starting.
I20260810 22:58:25.544392 7417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:25.544728 7417 log.cc:824] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:25.545521 7417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc: No bootstrap required, opened a new log
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:25.547761 7417 raft_consensus.cc:348] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } }
I20260810 22:58:25.547935 7417 raft_consensus.cc:374] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:25.547979 7417 raft_consensus.cc:729] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceb10485cd8844269e590f8f59bc9ccc, State: Initialized, Role: FOLLOWER
I20260810 22:58:25.548105 7417 consensus_queue.cc:260] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } }
I20260810 22:58:25.548190 7417 raft_consensus.cc:388] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:25.548213 7417 raft_consensus.cc:482] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:25.548302 7417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:25.548964 7417 raft_consensus.cc:504] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } }
I20260810 22:58:25.549057 7417 leader_election.cc:302] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ceb10485cd8844269e590f8f59bc9ccc; no voters:
I20260810 22:58:25.549279 7417 leader_election.cc:288] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:25.549324 7426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:25.549476 7426 raft_consensus.cc:686] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [term 1 LEADER]: Becoming Leader. State: Replica: ceb10485cd8844269e590f8f59bc9ccc, State: Running, Role: LEADER
I20260810 22:58:25.549564 7417 sys_catalog.cc:564] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:25.549569 7426 consensus_queue.cc:237] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } }
I20260810 22:58:25.550184 7430 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ceb10485cd8844269e590f8f59bc9ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } } }
I20260810 22:58:25.550271 7430 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:25.550427 7426 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ceb10485cd8844269e590f8f59bc9ccc. Latest consensus state: current_term: 1 leader_uuid: "ceb10485cd8844269e590f8f59bc9ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceb10485cd8844269e590f8f59bc9ccc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45541 } } }
I20260810 22:58:25.550489 7426 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:25.551333 7444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:25.551860 7444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:25.552179 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:25.541154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39538 (local address 127.6.47.190:45541)
0810 22:58:25.541353 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:25.541362 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:25.541395 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:58:25.541820 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:25.541826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:25.541868 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:25.542014 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:25.542676 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.543558 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.544373 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.544914 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.545027 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:25.547604 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:25.547607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:25.547608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:25.547617 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:25.549600 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.550037 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.550041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.550042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.550150 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.550443 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.550444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.550444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.550730 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:25.551719 (+ 989us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:25.551798 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 22:58:25.551957 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260810 22:58:25.553517 7444 catalog_manager.cc:1347] Generated new cluster ID: 8ad72a8e617b4acb934c2ffcd3e7c118
I20260810 22:58:25.553570 7444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:25.557273 7444 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:25.557869 7444 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:25.563541 7444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ceb10485cd8844269e590f8f59bc9ccc: Generated new TSK 0
I20260810 22:58:25.563730 7444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:25.600225 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:25.686844 7465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:25.687041 7465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:25.687067 7465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:25.688148 7465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:25.688184 7465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:25.688210 7465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:25.689726 7465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:25.689914 7465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:25.690091 7465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:25.694996 7474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:25.694996 7479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:25.695106 7476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:25.695258 7465 server_base.cc:1034] running on GCE node
I20260810 22:58:25.695482 7465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:25.695726 7465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:25.696880 7465 hybrid_clock.cc:648] HybridClock initialized: now 1786402705696877 us; error 35 us; skew 500 ppm
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:25.699239 7465 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:25.699679 7465 webserver.cc:460] Webserver started at http://127.6.47.129:45889/ using document root <none> and password file <none>
I20260810 22:58:25.699873 7465 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:25.699911 7465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:25.699999 7465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:25.701069 7465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "19b3c47320b54650a7ddc44b1bac471a"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "3c829daa010da42513c1c47445ba6179"
server_key_iv: "34bbcc78ffc78c9735c491db520f0138"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.701489 7465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "19b3c47320b54650a7ddc44b1bac471a"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "3c829daa010da42513c1c47445ba6179"
server_key_iv: "34bbcc78ffc78c9735c491db520f0138"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.702740 7465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:25.703428 7492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.703603 7465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:25.703740 7465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "19b3c47320b54650a7ddc44b1bac471a"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "3c829daa010da42513c1c47445ba6179"
server_key_iv: "34bbcc78ffc78c9735c491db520f0138"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.703811 7465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:25.717463 7465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:25.719772 7465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:25.719925 7465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:25.720191 7465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:25.720609 7465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:25.720664 7465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.720698 7465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:25.720726 7465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.733820 7465 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:35047
I20260810 22:58:25.733906 7733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:35047 every 8 connection(s)
I20260810 22:58:25.734277 7465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:25.737483 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7465
I20260810 22:58:25.737588 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:25.737747 6334 external_mini_cluster.cc:1428] Setting key 16a8b7802b278e0f39ebee5e6f904b53
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:25.746019 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:25.735132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50535 (local address 127.6.47.190:45541)
0810 22:58:25.735178 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:25.735180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:25.735533 (+ 353us) server_negotiation.cc:407] Connection header received
0810 22:58:25.736696 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:25.736698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:25.736709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:25.736772 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:25.737455 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.738128 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.739863 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.740176 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.740287 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:25.742539 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:25.742541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:25.742541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:25.742549 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:25.744370 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.744975 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.744976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.744976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.745030 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.745358 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.745359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.745359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.745464 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:25.745512 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:25.745896 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 22:58:25.745957 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:25.746281 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:25.735096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45541 (local address 127.6.47.129:50535)
0810 22:58:25.735292 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:25.735342 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:25.736574 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:25.736846 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:25.736851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:25.737115 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:25.737295 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:25.737299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.738296 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.738296 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:25.739733 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:25.739736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.740298 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.740299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:25.740449 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:25.741069 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:25.741077 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:25.742451 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:25.744467 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:25.744475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:25.744477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:25.744893 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:25.745105 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:25.745106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:25.745106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:25.745294 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:25.745533 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:25.745540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:25.745731 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 22:58:25.745867 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 22:58:25.746042 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260810 22:58:25.746913 7739 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45541
I20260810 22:58:25.747035 7739 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:25.747219 7739 heartbeater.cc:511] Master 127.6.47.190:45541 requested a full tablet report, sending...
I20260810 22:58:25.747870 7345 ts_manager.cc:194] Registered new tserver with Master: 19b3c47320b54650a7ddc44b1bac471a (127.6.47.129:35047)
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
I20260810 22:58:25.748840 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:50535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:25.767206 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:25.849548 7748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:25.849843 7748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:25.849916 7748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:25.851266 7748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:25.851329 7748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:25.851375 7748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:25.852917 7748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:25.853200 7748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:25.853428 7748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:25.858410 7760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:25.858457 7754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:25.858421 7757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:25.858680 7748 server_base.cc:1034] running on GCE node
I20260810 22:58:25.858873 7748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:25.859054 7748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:25.860193 7748 hybrid_clock.cc:648] HybridClock initialized: now 1786402705860177 us; error 30 us; skew 500 ppm
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:25.862636 7748 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:25.863090 7748 webserver.cc:460] Webserver started at http://127.6.47.130:40505/ using document root <none> and password file <none>
I20260810 22:58:25.863337 7748 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:25.863405 7748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:25.863543 7748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:25.864626 7748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "7fe4c480d7ff4c77af6fa56fde7148fb"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "307fae7c4ac1a31b06d9d4da54405b65"
server_key_iv: "fe9a1430d24f6ac032cb6bcce2f7bb23"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.865005 7748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "7fe4c480d7ff4c77af6fa56fde7148fb"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "307fae7c4ac1a31b06d9d4da54405b65"
server_key_iv: "fe9a1430d24f6ac032cb6bcce2f7bb23"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.866350 7748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:25.867069 7772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.867264 7748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:25.867377 7748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "7fe4c480d7ff4c77af6fa56fde7148fb"
format_stamp: "Formatted at 2026-08-10 22:58:25 on dist-test-slave-rwrg"
server_key: "307fae7c4ac1a31b06d9d4da54405b65"
server_key_iv: "fe9a1430d24f6ac032cb6bcce2f7bb23"
server_key_version: "encryptionkey@0"
I20260810 22:58:25.867458 7748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:25.882599 7748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:25.884370 7748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:25.884616 7748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:25.884855 7748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:25.885179 7748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:25.885236 7748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.885275 7748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:25.885304 7748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:25.891587 7748 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:45831
I20260810 22:58:25.891690 7991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:45831 every 8 connection(s)
I20260810 22:58:25.891986 7748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:25.892683 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7748
I20260810 22:58:25.892776 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:25.892971 6334 external_mini_cluster.cc:1428] Setting key 1a55845660eb89312cf3fef07e6a714f
Aug 10 22:58:25 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
I20260810 22:58:25.902643 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:25.892791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:47803 (local address 127.6.47.190:45541)
0810 22:58:25.892868 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:25.892870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:25.893048 (+ 178us) server_negotiation.cc:407] Connection header received
0810 22:58:25.894083 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:25.894084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:25.894097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:25.894173 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:25.894746 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.895523 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.897055 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.897254 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.897305 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:25.899630 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:25.899632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:25.899632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:25.899640 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:25.901190 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.901625 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.901627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.901627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.901692 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:25.901955 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:25.901956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:25.901957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:25.902092 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:25.902167 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:25.902453 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 22:58:25.902538 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:58:25.902822 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:25.892745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45541 (local address 127.6.47.130:47803)
0810 22:58:25.892912 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:25.892955 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:25.893979 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:25.894232 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:25.894235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:25.894478 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:25.894661 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:25.894665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.895620 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.895621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:25.896954 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:25.896957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:25.897335 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:25.897336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:25.897478 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:25.897981 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:25.897992 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:25.899528 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:25.901263 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:25.901271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:25.901273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:25.901541 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:25.901769 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:25.901769 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:25.901770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:25.901880 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:25.902172 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:25.902180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:25.902318 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:58:25.902430 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 22:58:25.902591 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260810 22:58:25.903357 7994 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45541
I20260810 22:58:25.903481 7994 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:25.903671 7994 heartbeater.cc:511] Master 127.6.47.190:45541 requested a full tablet report, sending...
I20260810 22:58:25.904076 7345 ts_manager.cc:194] Registered new tserver with Master: 7fe4c480d7ff4c77af6fa56fde7148fb (127.6.47.130:45831)
I20260810 22:58:25.904592 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:47803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:25.919124 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:26.001797 8000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:26.001991 8000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:26.002014 8000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:26.003175 8000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:26.003217 8000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:26.003268 8000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:26.004850 8000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45541
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:26.005102 8000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:26.005369 8000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:26.010500 8012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:26.010541 8009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:26.010615 8010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:26.010628 8000 server_base.cc:1034] running on GCE node
I20260810 22:58:26.010888 8000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:26.011121 8000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:26.012262 8000 hybrid_clock.cc:648] HybridClock initialized: now 1786402706012249 us; error 33 us; skew 500 ppm
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:26.014855 8000 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:26.015344 8000 webserver.cc:460] Webserver started at http://127.6.47.131:34153/ using document root <none> and password file <none>
I20260810 22:58:26.015599 8000 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:26.015645 8000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:26.015815 8000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:26.016964 8000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "5a20ce6dd6b940bda86f5499bbb59b2f"
format_stamp: "Formatted at 2026-08-10 22:58:26 on dist-test-slave-rwrg"
server_key: "a03f520990b987349c7f3a4bd18f8026"
server_key_iv: "458df8c0ce02b31e93f77e5b2f6a0d98"
server_key_version: "encryptionkey@0"
I20260810 22:58:26.017330 8000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "5a20ce6dd6b940bda86f5499bbb59b2f"
format_stamp: "Formatted at 2026-08-10 22:58:26 on dist-test-slave-rwrg"
server_key: "a03f520990b987349c7f3a4bd18f8026"
server_key_iv: "458df8c0ce02b31e93f77e5b2f6a0d98"
server_key_version: "encryptionkey@0"
I20260810 22:58:26.018748 8000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:26.019567 8023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:26.019773 8000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:26.019886 8000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "5a20ce6dd6b940bda86f5499bbb59b2f"
format_stamp: "Formatted at 2026-08-10 22:58:26 on dist-test-slave-rwrg"
server_key: "a03f520990b987349c7f3a4bd18f8026"
server_key_iv: "458df8c0ce02b31e93f77e5b2f6a0d98"
server_key_version: "encryptionkey@0"
I20260810 22:58:26.019963 8000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:26.045320 8000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:26.047235 8000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:26.047412 8000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:26.047657 8000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:26.047994 8000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:26.048051 8000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:26.048094 8000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:26.048135 8000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:26.055924 8000 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:37167
I20260810 22:58:26.055965 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:37167 every 8 connection(s)
I20260810 22:58:26.056317 8000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:26.064299 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8000
I20260810 22:58:26.064411 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:26.064610 6334 external_mini_cluster.cc:1428] Setting key 8a157823ba93ad1eb6551061fba5aa0c
I20260810 22:58:26.066202 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.057102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:58475 (local address 127.6.47.190:45541)
0810 22:58:26.057184 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.057187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.057434 (+ 247us) server_negotiation.cc:407] Connection header received
0810 22:58:26.058472 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.058473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.058486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.058543 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:26.059186 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.059668 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.060762 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.060959 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.061047 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.063166 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:26.063168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:26.063169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:26.063176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:26.064902 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.065392 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.065393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.065394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.065458 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.065692 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.065692 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.065693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.065795 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:26.065846 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.066073 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.066141 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 22:58:26.066398 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.057084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45541 (local address 127.6.47.131:58475)
0810 22:58:26.057309 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.057345 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.058309 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.058627 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.058631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.058893 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:26.059079 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.059083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.059764 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.059765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.060671 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.060674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.061060 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.061061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.061186 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.061725 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:26.061733 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:26.063072 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:26.064982 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:26.064990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:26.064992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:26.065296 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:26.065531 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:26.065532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:26.065532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:26.065632 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:26.065849 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:26.065854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:26.065962 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.066042 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.066156 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":145,"threads_started":1}
I20260810 22:58:26.066816 8234 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45541
I20260810 22:58:26.066912 8234 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:26.067049 8234 heartbeater.cc:511] Master 127.6.47.190:45541 requested a full tablet report, sending...
I20260810 22:58:26.067370 7345 ts_manager.cc:194] Registered new tserver with Master: 5a20ce6dd6b940bda86f5499bbb59b2f (127.6.47.131:37167)
I20260810 22:58:26.067693 7345 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:58475
I20260810 22:58:26.076181 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:58:26.083802 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.077858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.6.47.190:45541)
0810 22:58:26.077915 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.077917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.078029 (+ 112us) server_negotiation.cc:407] Connection header received
0810 22:58:26.078114 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.078115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.078125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.078189 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:26.078725 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.079224 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.079937 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.080145 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.080221 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.081499 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:26.081501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:26.081501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:26.081508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:26.082657 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.083003 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.083004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.083005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.083059 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.083266 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.083267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.083267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.083414 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:26.083517 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.083626 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.083701 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:58:26.086043 7345 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:26.087746 7345 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:26.094903 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:35047 (local address 127.0.0.1:57270)
0810 22:58:26.090422 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.090469 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.090549 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.091134 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.091138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.091140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:26.091216 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.091218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092661 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.092662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.093702 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.093705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094608 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.094696 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094710 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.094780 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.094832 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":42,"thread_start_us":60,"threads_started":1}
I20260810 22:58:26.094949 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.6.47.129:35047)
0810 22:58:26.090880 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.090886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.090897 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:26.090945 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.090948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.090966 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.091055 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:26.091481 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092538 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.093793 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094510 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094589 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094649 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.094817 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.094861 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":103,"threads_started":1}
I20260810 22:58:26.095119 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37167 (local address 127.0.0.1:55484)
0810 22:58:26.090708 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.090716 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.090752 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.091134 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.091138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.091142 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:26.091274 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.091277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092584 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.092585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.093948 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.093951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094880 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.094935 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094938 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.094987 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.095030 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":50,"thread_start_us":74,"threads_started":1}
I20260810 22:58:26.095139 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:45831 (local address 127.0.0.1:51424)
0810 22:58:26.090580 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.090592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.090644 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.091380 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.091381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.091383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:26.091446 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.091447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092464 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.092465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.093957 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.093959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094705 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.094784 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094789 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.094837 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.094883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":56,"thread_start_us":83,"threads_started":1}
I20260810 22:58:26.095431 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.6.47.131:37167)
0810 22:58:26.090932 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.090935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.090944 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:26.090978 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.090979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.090996 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.091063 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:26.091347 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092473 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094046 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094725 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094801 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094853 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.095317 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.095362 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":139,"threads_started":1}
I20260810 22:58:26.095443 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.090469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.6.47.130:45831)
0810 22:58:26.091235 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.091240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.091250 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:26.091290 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.091292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.091306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.091386 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:26.091527 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.092351 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094054 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.094547 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.094640 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.094700 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.094951 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.095151 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":107,"threads_started":1}
I20260810 22:58:26.095880 7592 tablet_service.cc:1467] Processing CreateTablet for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87 (DEFAULT_TABLE table=test-table [id=d4f8fb8c29b54e34ad485b8cbc16c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:26.096101 8117 tablet_service.cc:1467] Processing CreateTablet for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87 (DEFAULT_TABLE table=test-table [id=d4f8fb8c29b54e34ad485b8cbc16c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:26.096299 7592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:26.096279 7851 tablet_service.cc:1467] Processing CreateTablet for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87 (DEFAULT_TABLE table=test-table [id=d4f8fb8c29b54e34ad485b8cbc16c909]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:26.096465 8117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:26.096618 7851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:26.099694 8265 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Bootstrap starting.
I20260810 22:58:26.100692 8266 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Bootstrap starting.
I20260810 22:58:26.100943 8265 tablet_bootstrap.cc:654] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:26.101147 8267 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Bootstrap starting.
I20260810 22:58:26.102236 8265 log.cc:824] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:26.102356 8267 tablet_bootstrap.cc:654] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:26.102447 8266 tablet_bootstrap.cc:654] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:26.103261 8267 log.cc:824] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:26.103350 8266 log.cc:824] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:26.103463 8265 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: No bootstrap required, opened a new log
I20260810 22:58:26.103547 8265 ts_tablet_manager.cc:1397] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 22:58:26.104125 8267 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: No bootstrap required, opened a new log
I20260810 22:58:26.104223 8267 ts_tablet_manager.cc:1397] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:26.104236 8266 tablet_bootstrap.cc:492] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: No bootstrap required, opened a new log
I20260810 22:58:26.104302 8266 ts_tablet_manager.cc:1397] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 22:58:26.106016 8265 raft_consensus.cc:348] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106045 8266 raft_consensus.cc:348] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106204 8265 raft_consensus.cc:374] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:26.106225 8266 raft_consensus.cc:374] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:26.106240 8265 raft_consensus.cc:729] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b3c47320b54650a7ddc44b1bac471a, State: Initialized, Role: FOLLOWER
I20260810 22:58:26.106271 8266 raft_consensus.cc:729] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe4c480d7ff4c77af6fa56fde7148fb, State: Initialized, Role: FOLLOWER
I20260810 22:58:26.106392 8265 consensus_queue.cc:260] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106408 8266 consensus_queue.cc:260] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106406 8267 raft_consensus.cc:348] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106563 8267 raft_consensus.cc:374] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:26.106601 8267 raft_consensus.cc:729] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a20ce6dd6b940bda86f5499bbb59b2f, State: Initialized, Role: FOLLOWER
I20260810 22:58:26.106719 8265 ts_tablet_manager.cc:1428] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 22:58:26.106736 8266 ts_tablet_manager.cc:1428] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 22:58:26.106719 8267 consensus_queue.cc:260] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.106863 7994 heartbeater.cc:503] Master 127.6.47.190:45541 was elected leader, sending a full tablet report...
I20260810 22:58:26.107051 8267 ts_tablet_manager.cc:1428] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 22:58:26.107054 8234 heartbeater.cc:503] Master 127.6.47.190:45541 was elected leader, sending a full tablet report...
I20260810 22:58:26.107152 7739 heartbeater.cc:503] Master 127.6.47.190:45541 was elected leader, sending a full tablet report...
W20260810 22:58:26.142551 7995 tablet.cc:2369] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:26.235082 7743 tablet.cc:2369] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:26.280226 8281 raft_consensus.cc:482] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:26.280360 8281 raft_consensus.cc:504] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.280812 8281 leader_election.cc:288] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19b3c47320b54650a7ddc44b1bac471a (127.6.47.129:35047), 5a20ce6dd6b940bda86f5499bbb59b2f (127.6.47.131:37167)
I20260810 22:58:26.284477 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.281007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:35047 (local address 127.6.47.130:50347)
0810 22:58:26.281068 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.281081 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.281139 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.281341 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.281343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.281345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:26.281421 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.281423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.282279 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.282280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.283506 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.283509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.284222 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.284222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.284286 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.284289 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.284334 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.284395 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 22:58:26.284480 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.281084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:50347 (local address 127.6.47.129:35047)
0810 22:58:26.281144 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.281146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.281203 (+ 57us) server_negotiation.cc:407] Connection header received
0810 22:58:26.281236 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.281237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.281248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.281312 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:26.281496 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.282194 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.283603 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.284129 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.284217 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.284226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.284364 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.284407 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:58:26.284704 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.281139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37167 (local address 127.6.47.130:52265)
0810 22:58:26.281332 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:26.281342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:26.281400 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:26.281551 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:26.281552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:26.281553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:26.281659 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.281660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.282438 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.282438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.283662 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:26.283665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.284418 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.284419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:26.284486 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.284513 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 22:58:26.284568 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 22:58:26.284608 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260810 22:58:26.284713 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.281247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:52265 (local address 127.6.47.131:37167)
0810 22:58:26.281315 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.281317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.281406 (+ 89us) server_negotiation.cc:407] Connection header received
0810 22:58:26.281450 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.281451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.281476 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.281535 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:26.281736 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.282366 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.283744 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.284311 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.284396 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.284403 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.284579 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.284625 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:58:26.284909 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4dcd3a09b84799b6a16e2d1a4b5f87" candidate_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b3c47320b54650a7ddc44b1bac471a" is_pre_election: true
I20260810 22:58:26.285046 7648 raft_consensus.cc:2454] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fe4c480d7ff4c77af6fa56fde7148fb in term 0.
I20260810 22:58:26.285302 7778 leader_election.cc:302] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b3c47320b54650a7ddc44b1bac471a, 7fe4c480d7ff4c77af6fa56fde7148fb; no voters:
I20260810 22:58:26.285449 8281 raft_consensus.cc:2781] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:26.285398 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4dcd3a09b84799b6a16e2d1a4b5f87" candidate_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" is_pre_election: true
I20260810 22:58:26.285507 8281 raft_consensus.cc:482] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:26.285524 8281 raft_consensus.cc:3037] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:26.285571 8155 raft_consensus.cc:2454] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fe4c480d7ff4c77af6fa56fde7148fb in term 0.
I20260810 22:58:26.286360 8281 raft_consensus.cc:504] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.286504 8281 leader_election.cc:288] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [CANDIDATE]: Term 1 election: Requested vote from peers 19b3c47320b54650a7ddc44b1bac471a (127.6.47.129:35047), 5a20ce6dd6b940bda86f5499bbb59b2f (127.6.47.131:37167)
I20260810 22:58:26.286716 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4dcd3a09b84799b6a16e2d1a4b5f87" candidate_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f"
I20260810 22:58:26.286738 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4dcd3a09b84799b6a16e2d1a4b5f87" candidate_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b3c47320b54650a7ddc44b1bac471a"
I20260810 22:58:26.286813 7648 raft_consensus.cc:3037] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:26.286834 8155 raft_consensus.cc:3037] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:26.287528 7648 raft_consensus.cc:2454] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 19b3c47320b54650a7ddc44b1bac471a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fe4c480d7ff4c77af6fa56fde7148fb in term 1.
I20260810 22:58:26.287551 8155 raft_consensus.cc:2454] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fe4c480d7ff4c77af6fa56fde7148fb in term 1.
I20260810 22:58:26.287708 7778 leader_election.cc:302] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b3c47320b54650a7ddc44b1bac471a, 7fe4c480d7ff4c77af6fa56fde7148fb; no voters:
I20260810 22:58:26.287796 8281 raft_consensus.cc:2781] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:26.287950 8281 raft_consensus.cc:686] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [term 1 LEADER]: Becoming Leader. State: Replica: 7fe4c480d7ff4c77af6fa56fde7148fb, State: Running, Role: LEADER
I20260810 22:58:26.288069 8281 consensus_queue.cc:237] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } }
I20260810 22:58:26.288981 7342 catalog_manager.cc:5495] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 7fe4c480d7ff4c77af6fa56fde7148fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fe4c480d7ff4c77af6fa56fde7148fb (127.6.47.130). New cstate: current_term: 1 leader_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19b3c47320b54650a7ddc44b1bac471a" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 35047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a20ce6dd6b940bda86f5499bbb59b2f" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fe4c480d7ff4c77af6fa56fde7148fb" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45831 } health_report { overall_health: HEALTHY } } }
W20260810 22:58:26.307144 8235 tablet.cc:2369] T ca4dcd3a09b84799b6a16e2d1a4b5f87 P 5a20ce6dd6b940bda86f5499bbb59b2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:26.324642 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.316955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.6.47.129:35047)
0810 22:58:26.317027 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.317029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.317196 (+ 167us) server_negotiation.cc:407] Connection header received
0810 22:58:26.317299 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.317300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.317319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.317389 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:26.318029 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.318501 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.319216 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.319405 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.319480 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.321932 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:26.321943 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:26.321943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:26.321950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:26.323529 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.323892 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.323895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.323896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.323957 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.324152 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.324153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.324153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.324288 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:26.324404 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.324454 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.324576 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:26.349591 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.342547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.6.47.129:35047)
0810 22:58:26.342598 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.342600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.342650 (+ 50us) server_negotiation.cc:407] Connection header received
0810 22:58:26.342719 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.342721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.342734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.342794 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:26.343393 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.343879 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.344667 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.344851 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.344906 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.347060 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:26.347061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:26.347061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:26.347068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:26.348477 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.348829 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.348830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.348830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.348882 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.349115 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.349116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.349116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.349212 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:26.349301 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.349424 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.349485 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[7258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:26.373091 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:26.366059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.6.47.129:35047)
0810 22:58:26.366147 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:26.366149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:26.366252 (+ 103us) server_negotiation.cc:407] Connection header received
0810 22:58:26.366349 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:26.366350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:26.366360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:26.366434 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:26.366976 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.367414 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.368152 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:26.368372 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:26.368453 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:26.370682 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:26.370684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:26.370684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:26.370692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:26.371867 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.372225 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.372227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.372227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.372285 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:26.372537 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:26.372538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:26.372538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:26.372638 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:26.372767 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:26.372918 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 22:58:26.372980 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 22:58:26.373620 7543 tablet_service.cc:2908] Rejecting scan request for tablet ca4dcd3a09b84799b6a16e2d1a4b5f87: Uninitialized: clean time has not yet been initialized
I20260810 22:58:26.374496 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7465
I20260810 22:58:26.388217 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7748
I20260810 22:58:26.401769 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8000
I20260810 22:58:26.406804 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7284
2026-08-10T22:58:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2393 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[8311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[8311](info): set up 2 sockets
Aug 10 22:58:26 dist-test-slave-rwrg krb5kdc[8311](info): commencing operation
krb5kdc: starting...
W20260810 22:58:27.675817 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:27 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:27.799491 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44491
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:44491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:27.881978 8328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:27.882186 8328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:27.882208 8328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:27.883245 8328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:27.883276 8328 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:58:27.883289 8328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:27.883302 8328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:27.883313 8328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:27.883325 8328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:27.884684 8328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:44491
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:27.884869 8328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:27.885040 8328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:27.889856 8335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:27.889926 8334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:27.889865 8340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:27.889940 8328 server_base.cc:1034] running on GCE node
I20260810 22:58:27.890455 8328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:27.890750 8328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:27.891935 8328 hybrid_clock.cc:648] HybridClock initialized: now 1786402707891914 us; error 44 us; skew 500 ppm
Aug 10 22:58:27 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402707, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:27.899111 8328 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:27.899474 8328 webserver.cc:460] Webserver started at http://127.6.47.190:43699/ using document root <none> and password file <none>
I20260810 22:58:27.899693 8328 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:27.899735 8328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:27.899824 8328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:27.900974 8328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "2c9a08829e184192b694632dc3ac7fbd"
format_stamp: "Formatted at 2026-08-10 22:58:27 on dist-test-slave-rwrg"
server_key: "bad5dcc21f18c574db9e9f26133ca343"
server_key_iv: "80b276290fc900638b5afa273b751723"
server_key_version: "encryptionkey@0"
I20260810 22:58:27.901291 8328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "2c9a08829e184192b694632dc3ac7fbd"
format_stamp: "Formatted at 2026-08-10 22:58:27 on dist-test-slave-rwrg"
server_key: "bad5dcc21f18c574db9e9f26133ca343"
server_key_iv: "80b276290fc900638b5afa273b751723"
server_key_version: "encryptionkey@0"
I20260810 22:58:27.902563 8328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:27.903296 8358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:27.903528 8328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:27.903643 8328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2c9a08829e184192b694632dc3ac7fbd"
format_stamp: "Formatted at 2026-08-10 22:58:27 on dist-test-slave-rwrg"
server_key: "bad5dcc21f18c574db9e9f26133ca343"
server_key_iv: "80b276290fc900638b5afa273b751723"
server_key_version: "encryptionkey@0"
I20260810 22:58:27.903702 8328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:27.921983 8328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:27.924009 8328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:27.924118 8328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:27.929200 8328 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:44491
I20260810 22:58:27.929301 8473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:44491 every 8 connection(s)
I20260810 22:58:27.929746 8328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:27.930496 8475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:27.932947 8475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Bootstrap starting.
I20260810 22:58:27.933610 8475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:27.933943 8475 log.cc:824] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:27.934641 8475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: No bootstrap required, opened a new log
I20260810 22:58:27.934897 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8328
I20260810 22:58:27.935024 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:27.935197 6334 external_mini_cluster.cc:1428] Setting key 90fff6e83532ef5ef1b4b50c39168969
I20260810 22:58:27.937253 8475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } }
I20260810 22:58:27.937414 8475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:27.937479 8475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c9a08829e184192b694632dc3ac7fbd, State: Initialized, Role: FOLLOWER
I20260810 22:58:27.937683 8475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } }
I20260810 22:58:27.937803 8475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:27.937847 8475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:27.937896 8475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:27.938433 8475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } }
I20260810 22:58:27.938558 8475 leader_election.cc:302] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c9a08829e184192b694632dc3ac7fbd; no voters:
I20260810 22:58:27.938778 8475 leader_election.cc:288] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:27.938809 8485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:27.938993 8485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [term 1 LEADER]: Becoming Leader. State: Replica: 2c9a08829e184192b694632dc3ac7fbd, State: Running, Role: LEADER
I20260810 22:58:27.939133 8475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:27.939288 8485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } }
I20260810 22:58:27.939646 8485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c9a08829e184192b694632dc3ac7fbd. Latest consensus state: current_term: 1 leader_uuid: "2c9a08829e184192b694632dc3ac7fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } } }
I20260810 22:58:27.939735 8485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:27.940836 8486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c9a08829e184192b694632dc3ac7fbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c9a08829e184192b694632dc3ac7fbd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44491 } } }
I20260810 22:58:27.940948 8486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd [sys.catalog]: This master's current role is: LEADER
W20260810 22:58:27.941177 8519 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:58:27.941254 8519 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:58:27.941313 8504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:27.941478 8504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:27.943038 8504 catalog_manager.cc:1347] Generated new cluster ID: 8f15471de6184527b96e864c71dc3f74
I20260810 22:58:27.943096 8504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:27 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:27.947268 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:27.935762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.6.47.190:44491)
0810 22:58:27.935968 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:27.935978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:27.936016 (+ 38us) server_negotiation.cc:407] Connection header received
0810 22:58:27.937234 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:27.937242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:27.937291 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:27.937431 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:27.938212 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:27.939475 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:27.940488 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:27.941180 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:27.941297 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:27.943899 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:27.943903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:27.943904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:27.943916 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:27.945696 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:27.946074 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:27.946077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:27.946078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:27.946176 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:27.946385 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:27.946386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:27.946386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:27.946629 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:27.946794 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:27.946891 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 22:58:27.947069 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260810 22:58:27.972273 8504 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:27.972880 8504 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:27.978386 8504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Generated new TSK 0
I20260810 22:58:27.978569 8504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:28.013410 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44491
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:28.107730 8527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:28.107929 8527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:28.107951 8527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:28.109009 8527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:28.109041 8527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:28.109059 8527 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:28.109074 8527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:28.110481 8527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:44491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:28.110679 8527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:28.110854 8527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:28.115859 8536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:28.115893 8527 server_base.cc:1034] running on GCE node
W20260810 22:58:28.115836 8538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:28.115849 8534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:28.116370 8527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:28.116639 8527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:28.117842 8527 hybrid_clock.cc:648] HybridClock initialized: now 1786402708117819 us; error 36 us; skew 500 ppm
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:28.120390 8527 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:28.120864 8527 webserver.cc:460] Webserver started at http://127.6.47.129:38939/ using document root <none> and password file <none>
I20260810 22:58:28.121117 8527 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:28.121189 8527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:28.121341 8527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:28.122385 8527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "4ed0b5755d294544a3174edb06502790"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "25136e6b34dd2b48b1346f3260bf9c1d"
server_key_iv: "f4e2d3a63a9defd575aa534633b11409"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.122741 8527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "4ed0b5755d294544a3174edb06502790"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "25136e6b34dd2b48b1346f3260bf9c1d"
server_key_iv: "f4e2d3a63a9defd575aa534633b11409"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.124012 8527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:28.124713 8556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.124876 8527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:28.124992 8527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "4ed0b5755d294544a3174edb06502790"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "25136e6b34dd2b48b1346f3260bf9c1d"
server_key_iv: "f4e2d3a63a9defd575aa534633b11409"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.125068 8527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:28.148788 8527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:28.150692 8527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:28.150856 8527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:28.151381 8527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:28.151454 8527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.151499 8527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:28.151522 8527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.158605 8527 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:42123
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:28.158687 8797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:42123 every 8 connection(s)
I20260810 22:58:28.158972 8527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:28.159880 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8527
I20260810 22:58:28.159986 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:28.160178 6334 external_mini_cluster.cc:1428] Setting key 0f3944411ef701629b1e45184a95b637
I20260810 22:58:28.162320 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.152037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:58195 (local address 127.6.47.190:44491)
0810 22:58:28.152106 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.152108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.152288 (+ 180us) server_negotiation.cc:407] Connection header received
0810 22:58:28.152847 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.152849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.152865 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.152958 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.153795 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.154611 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.156114 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.156467 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.156591 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.159153 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.159155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.159155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.159162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.160905 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.161317 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.161319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.161319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.161380 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.161657 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.161658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.161658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.161775 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.161849 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.162114 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.162229 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 22:58:28.162515 8575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.151964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.129:58195)
0810 22:58:28.152160 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.152199 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.152740 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.153032 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.153035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.153392 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.153685 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.153693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.154718 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.154719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.156020 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.156023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.156622 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.156623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.156748 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.157341 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.157353 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.159057 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.160982 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.160990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.160992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.161222 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.161469 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.161469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.161470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.161579 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.161841 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.161848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.161983 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.162090 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.162270 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":117,"threads_started":1}
I20260810 22:58:28.163015 8800 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44491
I20260810 22:58:28.163134 8800 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:28.163328 8800 heartbeater.cc:511] Master 127.6.47.190:44491 requested a full tablet report, sending...
I20260810 22:58:28.163884 8386 ts_manager.cc:194] Registered new tserver with Master: 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123)
I20260810 22:58:28.164737 8386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:58195
I20260810 22:58:28.169992 8820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.164170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.129:49121)
0810 22:58:28.164334 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.164349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.164396 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.164538 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.164539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.164705 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.164756 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.164757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.165572 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.165573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.166075 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.166077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.166580 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.166581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.166647 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.167079 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.167083 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.167359 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.169026 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.169027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.169027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.169182 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.169419 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.169419 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.169420 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.169470 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.169751 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.169751 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.169802 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.169853 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.169926 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260810 22:58:28.170099 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.164244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:49121 (local address 127.6.47.190:44491)
0810 22:58:28.164287 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.164289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.164405 (+ 116us) server_negotiation.cc:407] Connection header received
0810 22:58:28.164429 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.164430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.164439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.164530 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.164829 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.165481 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.166162 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.166483 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.166582 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.167448 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.167449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.167450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.167457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.168952 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.169266 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.169267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.169268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.169347 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.169544 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.169545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.169545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.169683 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.169756 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.169909 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.169988 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:28.187511 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44491
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:28.269320 8823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:28.269549 8823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:28.269588 8823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:28.270726 8823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:28.270794 8823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:28.270829 8823 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:28.270870 8823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:28.272274 8823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:44491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:28.272547 8823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:28.272782 8823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:28.277680 8829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:28.277715 8837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:28.277765 8833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:28.277912 8823 server_base.cc:1034] running on GCE node
I20260810 22:58:28.278185 8823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:28.278440 8823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:28.279609 8823 hybrid_clock.cc:648] HybridClock initialized: now 1786402708279583 us; error 30 us; skew 500 ppm
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:28.282081 8823 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:28.282588 8823 webserver.cc:460] Webserver started at http://127.6.47.130:43473/ using document root <none> and password file <none>
I20260810 22:58:28.282817 8823 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:28.282886 8823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:28.283041 8823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:28.284088 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "552903190a424a15aa41c03667672e49"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "6c584e3ab61e4f3d7b7b784ebb60beda"
server_key_iv: "d510627ea83040e71a75e84c42eec533"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.284461 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "552903190a424a15aa41c03667672e49"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "6c584e3ab61e4f3d7b7b784ebb60beda"
server_key_iv: "d510627ea83040e71a75e84c42eec533"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.285876 8823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:28.286655 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.286832 8823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:28.286937 8823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "552903190a424a15aa41c03667672e49"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "6c584e3ab61e4f3d7b7b784ebb60beda"
server_key_iv: "d510627ea83040e71a75e84c42eec533"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.287024 8823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:28.296731 8823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:28.298331 8823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:28.298478 8823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:28.298930 8823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:28.298988 8823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.299032 8823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:28.299062 8823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:28.306399 8823 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40527
I20260810 22:58:28.306473 9056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40527 every 8 connection(s)
I20260810 22:58:28.306777 8823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:28.309689 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.299464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:43225 (local address 127.6.47.190:44491)
0810 22:58:28.299541 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.299544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.299693 (+ 149us) server_negotiation.cc:407] Connection header received
0810 22:58:28.300142 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.300144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.300158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.300254 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.301086 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.301920 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.303589 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.303921 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.303993 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.306613 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.306615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.306615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.306623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.308191 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.308681 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.308683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.308683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.308743 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.309002 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.309003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.309003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.309138 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.309211 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.309507 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.309589 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:28.309842 8869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.299393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.130:43225)
0810 22:58:28.299568 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.299607 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.300043 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.300331 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.300334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.300697 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.300974 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.300981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.302038 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.302039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.303475 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.303478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.304034 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.304035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.304190 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.304801 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.304809 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.306527 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.308295 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.308303 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.308305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.308585 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.308812 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.308813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.308814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.308927 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.309217 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.309225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.309364 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.309466 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.309621 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260810 22:58:28.310325 9057 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44491
I20260810 22:58:28.310406 9057 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:28.310601 9057 heartbeater.cc:511] Master 127.6.47.190:44491 requested a full tablet report, sending...
I20260810 22:58:28.310999 8387 ts_manager.cc:194] Registered new tserver with Master: 552903190a424a15aa41c03667672e49 (127.6.47.130:40527)
I20260810 22:58:28.311549 8387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:43225
I20260810 22:58:28.312908 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8823
I20260810 22:58:28.313001 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:28.313184 6334 external_mini_cluster.cc:1428] Setting key 467264109c34651751515264914a94f0
I20260810 22:58:28.317533 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.311431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:38099 (local address 127.6.47.190:44491)
0810 22:58:28.311466 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.311468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.311559 (+ 91us) server_negotiation.cc:407] Connection header received
0810 22:58:28.311636 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.311637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.311647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.311708 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.312037 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.312659 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.313577 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.313819 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.313892 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.314705 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.314706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.314707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.314714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.316214 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.316612 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.316613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.316613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.316682 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.316891 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.316892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.316892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.317102 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.317192 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.317360 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.317447 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:28.317548 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.311352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.130:38099)
0810 22:58:28.311504 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.311513 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.311566 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.311712 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.311712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.311870 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.311959 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.311961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.312766 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.312768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.313509 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.313512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.313894 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.313895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.313970 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.314365 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.314368 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.314613 (+ 245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.316315 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.316316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.316316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.316477 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.316751 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.316751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.316752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.316806 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.317189 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.317190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.317241 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.317333 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.317449 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:28.341296 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44491
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:28.423956 9070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:28.424175 9070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:28.424213 9070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:28.425360 9070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:28.425429 9070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:28.425465 9070 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:28.425526 9070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:28.427102 9070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:44491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:28.427366 9070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:28.427659 9070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:58:28.432965 9070 server_base.cc:1034] running on GCE node
W20260810 22:58:28.432976 9082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:28.433080 9084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:28.432977 9081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:28.433365 9070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:28.433554 9070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:28.434695 9070 hybrid_clock.cc:648] HybridClock initialized: now 1786402708434683 us; error 28 us; skew 500 ppm
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:28.437328 9070 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:28.437772 9070 webserver.cc:460] Webserver started at http://127.6.47.131:35821/ using document root <none> and password file <none>
I20260810 22:58:28.438011 9070 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:28.438079 9070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:28.438233 9070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:28.439311 9070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "9720c8f8630444e9821a69e116c02b5b"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "eaaf6e53fc522993f1b3eae39fb2b3ca"
server_key_iv: "a79f46ea01e8d33fbdf5de38177c5e1d"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.439751 9070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "9720c8f8630444e9821a69e116c02b5b"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "eaaf6e53fc522993f1b3eae39fb2b3ca"
server_key_iv: "a79f46ea01e8d33fbdf5de38177c5e1d"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.441251 9070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:28.442019 9096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.442188 9070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:28.442270 9070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "9720c8f8630444e9821a69e116c02b5b"
format_stamp: "Formatted at 2026-08-10 22:58:28 on dist-test-slave-rwrg"
server_key: "eaaf6e53fc522993f1b3eae39fb2b3ca"
server_key_iv: "a79f46ea01e8d33fbdf5de38177c5e1d"
server_key_version: "encryptionkey@0"
I20260810 22:58:28.442345 9070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:28.453583 9070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:28.455937 9070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:28.456096 9070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:28.456692 9070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:28.456758 9070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.456796 9070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:28.456820 9070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:28.464231 9070 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:36557
I20260810 22:58:28.464327 9286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:36557 every 8 connection(s)
I20260810 22:58:28.464692 9070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:28.467007 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9070
I20260810 22:58:28.467100 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:28.467273 6334 external_mini_cluster.cc:1428] Setting key c0854479d67803b9db99c0c9b59899e0
I20260810 22:58:28.467536 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.457114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.131:49165)
0810 22:58:28.457276 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.457316 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.457784 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.458033 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.458036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.458257 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.458507 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.458512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.459561 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.459562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.461006 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.461010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.461588 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.461589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.461740 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.462367 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.462377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.464320 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.466126 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.466132 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.466134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.466344 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.466526 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.466527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.466527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.466622 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.466862 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.466868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.467028 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.467165 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.467333 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260810 22:58:28.467892 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.457122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:49165 (local address 127.6.47.190:44491)
0810 22:58:28.457208 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.457210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.457414 (+ 204us) server_negotiation.cc:407] Connection header received
0810 22:58:28.457882 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.457884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.457898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.457984 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.458585 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.459423 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.461109 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.461480 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.461567 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.464407 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.464409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.464409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.464418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.466030 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.466416 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.466417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.466417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.466469 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.466679 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.466680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.466680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.466782 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.466833 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.467704 (+ 871us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.467800 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:58:28.468286 9287 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44491
I20260810 22:58:28.468397 9287 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:28.468602 9287 heartbeater.cc:511] Master 127.6.47.190:44491 requested a full tablet report, sending...
I20260810 22:58:28.468978 8387 ts_manager.cc:194] Registered new tserver with Master: 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557)
I20260810 22:58:28.469575 8387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:49165
I20260810 22:58:28.474747 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.469424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:39925 (local address 127.6.47.190:44491)
0810 22:58:28.469471 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.469472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.469539 (+ 67us) server_negotiation.cc:407] Connection header received
0810 22:58:28.469611 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.469612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.469622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.469677 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.470050 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.470525 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.471240 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.471446 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.471522 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.472523 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.472525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.472525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.472532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.473667 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.474084 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.474086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.474086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.474161 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.474370 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.474371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.474372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.474469 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.474523 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.474635 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.474690 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:58:28.474772 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.469352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.6.47.131:39925)
0810 22:58:28.469484 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.469496 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.469554 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.469683 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.469683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.469891 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:28.469960 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.469962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.470633 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.470634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.471155 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.471156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.471522 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.471523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.471600 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.472097 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:28.472102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:28.472396 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:28.473750 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.473751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.473752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.474003 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.474236 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:28.474237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:28.474237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:28.474302 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:28.474528 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:28.474528 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:28.474570 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.474620 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.474699 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260810 22:58:28.478932 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:58:28 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:28.502094 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.495654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.6.47.190:44491)
0810 22:58:28.495722 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.495724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.495733 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:28.495764 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.495765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.495773 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.495822 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:28.496346 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.496902 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.497601 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.497771 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.497820 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.499859 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:28.499860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:28.499861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:28.499867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:28.501135 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.501449 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.501450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.501450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.501496 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:28.501657 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:28.501658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:28.501658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:28.501752 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:28.501825 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.501931 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.501989 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:28.503661 8387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:28.505321 8387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:28.512362 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.508049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:36557 (local address 127.0.0.1:36548)
0810 22:58:28.508291 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.508300 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.508346 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.508526 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.508527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.508529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.508633 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.508635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509867 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.509868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.511156 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.511159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.512049 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.512109 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.512120 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.512207 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.512261 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":68,"thread_start_us":69,"threads_started":1}
I20260810 22:58:28.512378 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.508171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.6.47.131:36557)
0810 22:58:28.508357 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.508362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.508372 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:28.508419 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.508422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.508438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.508534 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.508717 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509784 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.511233 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.511969 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512050 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.512110 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.512228 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.512286 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260810 22:58:28.512902 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.507781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41398 (local address 127.6.47.130:40527)
0810 22:58:28.508039 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.508046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.508057 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:28.508099 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.508101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.508124 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.508197 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.508415 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509680 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.511157 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.511977 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512050 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.512119 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.512750 (+ 631us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.512793 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
I20260810 22:58:28.512847 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.507595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40527 (local address 127.0.0.1:41398)
0810 22:58:28.507715 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.507758 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.507812 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.508216 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.508219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.508222 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.508328 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.508330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509766 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.509768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.511061 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.511064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.512591 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.512667 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.512671 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.512729 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.512771 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":93,"thread_start_us":51,"threads_started":1}
I20260810 22:58:28.513406 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.507876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:42123 (local address 127.0.0.1:38898)
0810 22:58:28.508175 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.508187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.508240 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.508837 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.508838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.508841 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.508885 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.508887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509810 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.509811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.511155 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.511159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.513029 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.513030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.513093 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.513096 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.513286 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.513328 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":57,"thread_start_us":64,"threads_started":1}
I20260810 22:58:28.513695 9186 tablet_service.cc:1467] Processing CreateTablet for tablet 3e598d31affa4ff388332563351b8929 (DEFAULT_TABLE table=test-table [id=64ea19c9be404bc1a40deec62708d34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.514047 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e598d31affa4ff388332563351b8929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.513943 8942 tablet_service.cc:1467] Processing CreateTablet for tablet 3e598d31affa4ff388332563351b8929 (DEFAULT_TABLE table=test-table [id=64ea19c9be404bc1a40deec62708d34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.514300 8942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e598d31affa4ff388332563351b8929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.516875 9331 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Bootstrap starting.
I20260810 22:58:28.517194 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.507990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.6.47.129:42123)
0810 22:58:28.508670 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.508674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.508684 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:28.508720 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.508722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.508743 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.508828 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.508940 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.509715 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512152 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.512899 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.512967 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.513031 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.517040 (+ 4009us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.517090 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":95,"threads_started":1}
I20260810 22:58:28.517879 8663 tablet_service.cc:1467] Processing CreateTablet for tablet 3e598d31affa4ff388332563351b8929 (DEFAULT_TABLE table=test-table [id=64ea19c9be404bc1a40deec62708d34f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.518134 9331 tablet_bootstrap.cc:654] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.518261 8663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e598d31affa4ff388332563351b8929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.518500 9331 log.cc:824] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:28.519444 9331 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: No bootstrap required, opened a new log
I20260810 22:58:28.519523 9331 ts_tablet_manager.cc:1397] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:28.521791 9331 raft_consensus.cc:348] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.521976 9331 raft_consensus.cc:374] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.522017 9331 raft_consensus.cc:729] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720c8f8630444e9821a69e116c02b5b, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.522120 9337 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Bootstrap starting.
I20260810 22:58:28.522177 9331 consensus_queue.cc:260] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.522289 9334 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Bootstrap starting.
I20260810 22:58:28.522477 9331 ts_tablet_manager.cc:1428] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:28.522637 9287 heartbeater.cc:503] Master 127.6.47.190:44491 was elected leader, sending a full tablet report...
I20260810 22:58:28.522766 9337 tablet_bootstrap.cc:654] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.523011 9334 tablet_bootstrap.cc:654] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.523115 9337 log.cc:824] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:28.523342 9334 log.cc:824] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:28.523945 9337 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: No bootstrap required, opened a new log
I20260810 22:58:28.524020 9337 ts_tablet_manager.cc:1397] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:28.524232 9334 tablet_bootstrap.cc:492] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: No bootstrap required, opened a new log
I20260810 22:58:28.524312 9334 ts_tablet_manager.cc:1397] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:28.525458 9337 raft_consensus.cc:348] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.525581 9337 raft_consensus.cc:374] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.525602 9337 raft_consensus.cc:729] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed0b5755d294544a3174edb06502790, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.525712 9337 consensus_queue.cc:260] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.525779 9334 raft_consensus.cc:348] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.525952 9334 raft_consensus.cc:374] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.525985 9337 ts_tablet_manager.cc:1428] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:28.525998 9334 raft_consensus.cc:729] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552903190a424a15aa41c03667672e49, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.526028 8800 heartbeater.cc:503] Master 127.6.47.190:44491 was elected leader, sending a full tablet report...
I20260810 22:58:28.526110 9334 consensus_queue.cc:260] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.526450 9334 ts_tablet_manager.cc:1428] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:28.526508 9057 heartbeater.cc:503] Master 127.6.47.190:44491 was elected leader, sending a full tablet report...
W20260810 22:58:28.557380 9058 tablet.cc:2369] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:28.659780 8801 tablet.cc:2369] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:28.715301 9288 tablet.cc:2369] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:28.727895 9344 raft_consensus.cc:482] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:28.728080 9344 raft_consensus.cc:504] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.728451 9344 leader_election.cc:288] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123), 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557)
I20260810 22:58:28.732522 8869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.728699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:42123 (local address 127.6.47.130:51837)
0810 22:58:28.728761 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.728774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.728867 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.729134 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.729135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.729138 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.729214 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.729217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.730188 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.730189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.731476 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.731478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.732211 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.732212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.732309 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.732310 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.732365 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.732413 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":17}
I20260810 22:58:28.732585 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.729049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:50899 (local address 127.6.47.131:36557)
0810 22:58:28.729117 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.729119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.729150 (+ 31us) server_negotiation.cc:407] Connection header received
0810 22:58:28.729221 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.729222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.729230 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.729331 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.729496 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.730213 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.731587 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.732122 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.732201 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.732209 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.732439 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.732483 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:58:28.732529 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.728907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:36557 (local address 127.6.47.130:50899)
0810 22:58:28.729090 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.729099 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.729160 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.729354 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.729354 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.729356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.729430 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.729431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.730280 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.730280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.731474 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.731478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.732212 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.732212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.732306 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.732310 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.732406 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.732434 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":17,"thread_start_us":103,"threads_started":1}
I20260810 22:58:28.732730 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.728743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:51837 (local address 127.6.47.129:42123)
0810 22:58:28.728805 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.728807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.728862 (+ 55us) server_negotiation.cc:407] Connection header received
0810 22:58:28.729008 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.729009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.729019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.729109 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.729322 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.730107 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.731589 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.732122 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.732200 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.732215 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.732576 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.732642 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:58:28.732964 9218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e598d31affa4ff388332563351b8929" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720c8f8630444e9821a69e116c02b5b" is_pre_election: true
I20260810 22:58:28.733002 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e598d31affa4ff388332563351b8929" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790" is_pre_election: true
I20260810 22:58:28.733136 9218 raft_consensus.cc:2454] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 0.
I20260810 22:58:28.733156 8681 raft_consensus.cc:2454] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 0.
I20260810 22:58:28.733389 8855 leader_election.cc:302] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552903190a424a15aa41c03667672e49, 9720c8f8630444e9821a69e116c02b5b; no voters:
I20260810 22:58:28.733523 9344 raft_consensus.cc:2781] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:28.733582 9344 raft_consensus.cc:482] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:28.733608 9344 raft_consensus.cc:3037] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.734678 9344 raft_consensus.cc:504] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.734809 9344 leader_election.cc:288] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 election: Requested vote from peers 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123), 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557)
I20260810 22:58:28.734966 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e598d31affa4ff388332563351b8929" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790"
I20260810 22:58:28.734994 9218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e598d31affa4ff388332563351b8929" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720c8f8630444e9821a69e116c02b5b"
I20260810 22:58:28.735076 8681 raft_consensus.cc:3037] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.735080 9218 raft_consensus.cc:3037] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.735795 9218 raft_consensus.cc:2454] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 1.
I20260810 22:58:28.735898 8681 raft_consensus.cc:2454] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 1.
I20260810 22:58:28.735961 8855 leader_election.cc:302] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552903190a424a15aa41c03667672e49, 9720c8f8630444e9821a69e116c02b5b; no voters:
I20260810 22:58:28.736055 9344 raft_consensus.cc:2781] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:28.736209 9344 raft_consensus.cc:686] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 1 LEADER]: Becoming Leader. State: Replica: 552903190a424a15aa41c03667672e49, State: Running, Role: LEADER
I20260810 22:58:28.736312 9344 consensus_queue.cc:237] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.737172 8387 catalog_manager.cc:5495] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 552903190a424a15aa41c03667672e49 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:28.795288 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.792075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41400 (local address 127.6.47.130:40527)
0810 22:58:28.792142 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.792145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.792157 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:58:28.792233 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.792234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.792260 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.792433 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:28.792612 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.793436 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.794203 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.794560 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.794631 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.794840 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:28.794999 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:28.795099 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.795131 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.795158 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:58:28.797228 9218 raft_consensus.cc:1261] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Refusing update from remote peer 552903190a424a15aa41c03667672e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:28.797264 8681 raft_consensus.cc:1261] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Refusing update from remote peer 552903190a424a15aa41c03667672e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:28.797590 9344 consensus_queue.cc:1035] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:28.797768 9347 consensus_queue.cc:1035] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:28.801000 9356 mvcc.cc:205] Tried to move back new op lower bound from 7317105495230402560 to 7317105494984409088. Current Snapshot: MvccSnapshot[applied={T|T < 7317105495230402560}]
I20260810 22:58:28.801344 9355 mvcc.cc:205] Tried to move back new op lower bound from 7317105495230402560 to 7317105494984409088. Current Snapshot: MvccSnapshot[applied={T|T < 7317105495230402560 or (T in {7317105495230402560})}]
I20260810 22:58:28.804059 8387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:28.804163 8387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:28.804944 8387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Sending DeleteTablet for 3 replicas of tablet 3e598d31affa4ff388332563351b8929
I20260810 22:58:28.805378 8942 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e598d31affa4ff388332563351b8929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:28 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:41398
I20260810 22:58:28.805405 9186 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e598d31affa4ff388332563351b8929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:28 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:36548
I20260810 22:58:28.805719 8663 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e598d31affa4ff388332563351b8929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:28 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:38898
I20260810 22:58:28.805897 8387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 22:58:28.806041 9367 tablet_replica.cc:330] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: stopping tablet replica
W20260810 22:58:28.806072 8387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:28.806130 9367 raft_consensus.cc:2229] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:28.806279 9367 raft_consensus.cc:2258] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:28.806864 9367 ts_tablet_manager.cc:1905] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:28.806907 9358 mvcc.cc:205] Tried to move back new op lower bound from 7317105495230402560 to 7317105494984409088. Current Snapshot: MvccSnapshot[applied={T|T < 7317105495230402560}]
I20260810 22:58:28.807763 8942 tablet_service.cc:1467] Processing CreateTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 (DEFAULT_TABLE table=test-table [id=215c2c9828cc43e68d7eea662e75a656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.807878 8942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c408ae6c4f0a4c51a0a0764e40769145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.808593 9368 tablet_replica.cc:330] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: stopping tablet replica
I20260810 22:58:28.808647 9186 tablet_service.cc:1467] Processing CreateTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 (DEFAULT_TABLE table=test-table [id=215c2c9828cc43e68d7eea662e75a656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.808708 9368 raft_consensus.cc:2229] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:28.808832 9368 raft_consensus.cc:2258] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:28.809537 9334 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Bootstrap starting.
I20260810 22:58:28.809751 9368 ts_tablet_manager.cc:1905] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:28.810004 9334 tablet_bootstrap.cc:654] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.807651 8663 tablet_service.cc:1467] Processing CreateTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 (DEFAULT_TABLE table=test-table [id=215c2c9828cc43e68d7eea662e75a656]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:28.810448 8663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c408ae6c4f0a4c51a0a0764e40769145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.811847 9337 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Bootstrap starting.
I20260810 22:58:28.811929 9368 ts_tablet_manager.cc:1918] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:28.812000 9368 log.cc:1196] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/3e598d31affa4ff388332563351b8929
I20260810 22:58:28.812207 9337 tablet_bootstrap.cc:654] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.812283 9368 ts_tablet_manager.cc:1939] T 3e598d31affa4ff388332563351b8929 P 552903190a424a15aa41c03667672e49: Deleting consensus metadata
I20260810 22:58:28.812690 8362 catalog_manager.cc:4841] TS 552903190a424a15aa41c03667672e49 (127.6.47.130:40527): tablet 3e598d31affa4ff388332563351b8929 (table test-table [id=64ea19c9be404bc1a40deec62708d34f]) successfully deleted
I20260810 22:58:28.812981 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c408ae6c4f0a4c51a0a0764e40769145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:28.814221 9331 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Bootstrap starting.
I20260810 22:58:28.814422 9367 ts_tablet_manager.cc:1918] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:28.814536 9367 log.cc:1196] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/3e598d31affa4ff388332563351b8929
I20260810 22:58:28.814687 9331 tablet_bootstrap.cc:654] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:28.814874 9367 ts_tablet_manager.cc:1939] T 3e598d31affa4ff388332563351b8929 P 9720c8f8630444e9821a69e116c02b5b: Deleting consensus metadata
I20260810 22:58:28.815297 8362 catalog_manager.cc:4841] TS 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557): tablet 3e598d31affa4ff388332563351b8929 (table test-table [id=64ea19c9be404bc1a40deec62708d34f]) successfully deleted
I20260810 22:58:28.815521 9337 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: No bootstrap required, opened a new log
I20260810 22:58:28.815577 9337 ts_tablet_manager.cc:1397] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 22:58:28.815747 9337 raft_consensus.cc:348] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.815816 9337 raft_consensus.cc:374] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.815837 9337 raft_consensus.cc:729] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed0b5755d294544a3174edb06502790, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.815888 9337 consensus_queue.cc:260] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.816007 9337 ts_tablet_manager.cc:1428] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.816522 9334 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: No bootstrap required, opened a new log
I20260810 22:58:28.816614 9334 ts_tablet_manager.cc:1397] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260810 22:58:28.816773 9334 raft_consensus.cc:348] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.816835 9334 raft_consensus.cc:374] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.816854 9334 raft_consensus.cc:729] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552903190a424a15aa41c03667672e49, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.816895 9334 consensus_queue.cc:260] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.816972 9334 ts_tablet_manager.cc:1428] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:28.818490 9331 tablet_bootstrap.cc:492] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: No bootstrap required, opened a new log
I20260810 22:58:28.818560 9331 ts_tablet_manager.cc:1397] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 22:58:28.818737 9331 raft_consensus.cc:348] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.818822 9331 raft_consensus.cc:374] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:28.818866 9331 raft_consensus.cc:729] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720c8f8630444e9821a69e116c02b5b, State: Initialized, Role: FOLLOWER
I20260810 22:58:28.818919 9331 consensus_queue.cc:260] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.819017 9331 ts_tablet_manager.cc:1428] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:28.820601 9370 tablet_replica.cc:330] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: stopping tablet replica
I20260810 22:58:28.820731 9370 raft_consensus.cc:2229] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:28.820811 9370 raft_consensus.cc:2258] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:28.821923 9370 ts_tablet_manager.cc:1905] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:28.823782 9370 ts_tablet_manager.cc:1918] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:28.823865 9370 log.cc:1196] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/3e598d31affa4ff388332563351b8929
I20260810 22:58:28.824127 9370 ts_tablet_manager.cc:1939] T 3e598d31affa4ff388332563351b8929 P 4ed0b5755d294544a3174edb06502790: Deleting consensus metadata
I20260810 22:58:28.824579 8362 catalog_manager.cc:4841] TS 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123): tablet 3e598d31affa4ff388332563351b8929 (table test-table [id=64ea19c9be404bc1a40deec62708d34f]) successfully deleted
I20260810 22:58:28.947971 9339 raft_consensus.cc:482] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:28.948071 9339 raft_consensus.cc:504] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.948479 9339 leader_election.cc:288] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123), 552903190a424a15aa41c03667672e49 (127.6.47.130:40527)
I20260810 22:58:28.951510 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.948824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:34177 (local address 127.6.47.129:42123)
0810 22:58:28.948912 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.948915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.948928 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:58:28.948981 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.948982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.948993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.949044 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.949216 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.949704 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.950718 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.951182 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.951236 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.951243 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.951429 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.951457 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 22:58:28.951562 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.948740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:42123 (local address 127.6.47.131:34177)
0810 22:58:28.948811 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.948827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.948898 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.949066 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.949068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.949070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.949143 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.949146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.949784 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.949785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.950630 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.950633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.951272 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.951273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.951350 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.951355 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.951415 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.951455 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 22:58:28.951865 9385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.948879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40527 (local address 127.6.47.131:59421)
0810 22:58:28.949004 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:28.949032 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:28.949082 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:28.949271 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:28.949272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:28.949273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:28.949345 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.949347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.950072 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.950073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.950910 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:28.950912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.951572 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.951573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:28.951664 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.951669 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:28.951751 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 22:58:28.951787 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260810 22:58:28.951886 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:28.949080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:59421 (local address 127.6.47.130:40527)
0810 22:58:28.949145 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:28.949148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:28.949156 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:28.949191 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:28.949193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:28.949202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:28.949269 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:28.949425 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.949985 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.951007 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:28.951490 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:28.951558 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:28.951565 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:28.951774 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 22:58:28.951820 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:58:28.951855 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c408ae6c4f0a4c51a0a0764e40769145" candidate_uuid: "9720c8f8630444e9821a69e116c02b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790" is_pre_election: true
I20260810 22:58:28.951953 8681 raft_consensus.cc:2454] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9720c8f8630444e9821a69e116c02b5b in term 0.
I20260810 22:58:28.952185 9099 leader_election.cc:302] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed0b5755d294544a3174edb06502790, 9720c8f8630444e9821a69e116c02b5b; no voters:
I20260810 22:58:28.952246 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c408ae6c4f0a4c51a0a0764e40769145" candidate_uuid: "9720c8f8630444e9821a69e116c02b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "552903190a424a15aa41c03667672e49" is_pre_election: true
I20260810 22:58:28.952359 9339 raft_consensus.cc:2781] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:28.952394 8980 raft_consensus.cc:2454] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9720c8f8630444e9821a69e116c02b5b in term 0.
I20260810 22:58:28.952412 9339 raft_consensus.cc:482] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:28.952435 9339 raft_consensus.cc:3037] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.953258 9339 raft_consensus.cc:504] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.953403 9339 leader_election.cc:288] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [CANDIDATE]: Term 1 election: Requested vote from peers 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123), 552903190a424a15aa41c03667672e49 (127.6.47.130:40527)
I20260810 22:58:28.953565 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c408ae6c4f0a4c51a0a0764e40769145" candidate_uuid: "9720c8f8630444e9821a69e116c02b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790"
I20260810 22:58:28.953608 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c408ae6c4f0a4c51a0a0764e40769145" candidate_uuid: "9720c8f8630444e9821a69e116c02b5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "552903190a424a15aa41c03667672e49"
I20260810 22:58:28.953662 8681 raft_consensus.cc:3037] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.953704 8980 raft_consensus.cc:3037] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:28.954468 8681 raft_consensus.cc:2454] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9720c8f8630444e9821a69e116c02b5b in term 1.
I20260810 22:58:28.954617 8980 raft_consensus.cc:2454] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9720c8f8630444e9821a69e116c02b5b in term 1.
I20260810 22:58:28.954661 9099 leader_election.cc:302] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed0b5755d294544a3174edb06502790, 9720c8f8630444e9821a69e116c02b5b; no voters:
I20260810 22:58:28.954764 9339 raft_consensus.cc:2781] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:28.954872 9339 raft_consensus.cc:686] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 1 LEADER]: Becoming Leader. State: Replica: 9720c8f8630444e9821a69e116c02b5b, State: Running, Role: LEADER
I20260810 22:58:28.954978 9339 consensus_queue.cc:237] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } }
I20260810 22:58:28.955700 8387 catalog_manager.cc:5495] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9720c8f8630444e9821a69e116c02b5b (127.6.47.131). New cstate: current_term: 1 leader_uuid: "9720c8f8630444e9821a69e116c02b5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } health_report { overall_health: HEALTHY } } }
Aug 10 22:58:29 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402707, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:29.032371 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.025243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44491 (local address 127.0.0.1:55872)
0810 22:58:29.025413 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:29.025426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:29.025529 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:29.025712 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:29.025713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:29.026058 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:29.026106 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.026108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.026793 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.026794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.027343 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.027345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.027700 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.027702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.027754 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.028307 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:29.028312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:29.029817 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:29.031249 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:29.031257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:29.031259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:29.031468 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:29.031688 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:29.031689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:29.031689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:29.031750 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:29.032003 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:29.032011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:29.032058 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:58:29.032133 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 22:58:29.032252 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260810 22:58:29.032382 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.025317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.6.47.190:44491)
0810 22:58:29.025423 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.025426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.025504 (+ 78us) server_negotiation.cc:407] Connection header received
0810 22:58:29.025603 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.025604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.025617 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.025707 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:29.026204 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.026706 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.027429 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.027621 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.027699 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.029914 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:29.029915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:29.029916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:29.029923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:29.031154 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.031557 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.031558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.031558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.031619 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.031821 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.031821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.031822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.031930 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:29.032000 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.032181 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.032302 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260810 22:58:29.034569 8387 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:55872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:58:29.034878 8387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:29.036567 8942 tablet_service.cc:1467] Processing CreateTablet for tablet 621bc7b7eec2487492ce14e57dd06201 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=69d0c904420e4ae098df2bf1bd1f8e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:29.036603 8663 tablet_service.cc:1467] Processing CreateTablet for tablet 621bc7b7eec2487492ce14e57dd06201 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=69d0c904420e4ae098df2bf1bd1f8e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:29.036603 9186 tablet_service.cc:1467] Processing CreateTablet for tablet 621bc7b7eec2487492ce14e57dd06201 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=69d0c904420e4ae098df2bf1bd1f8e3c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:29.036758 8942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621bc7b7eec2487492ce14e57dd06201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:29.036767 8663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621bc7b7eec2487492ce14e57dd06201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:29.036767 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621bc7b7eec2487492ce14e57dd06201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:29.038060 9334 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: Bootstrap starting.
I20260810 22:58:29.038316 9331 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: Bootstrap starting.
I20260810 22:58:29.038341 9337 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: Bootstrap starting.
I20260810 22:58:29.038439 9334 tablet_bootstrap.cc:654] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:29.038771 9337 tablet_bootstrap.cc:654] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:29.038777 9331 tablet_bootstrap.cc:654] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:29.039083 9334 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: No bootstrap required, opened a new log
I20260810 22:58:29.039129 9334 ts_tablet_manager.cc:1397] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:29.039278 9334 raft_consensus.cc:348] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.039371 9334 raft_consensus.cc:374] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:29.039397 9334 raft_consensus.cc:729] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552903190a424a15aa41c03667672e49, State: Initialized, Role: FOLLOWER
I20260810 22:58:29.039453 9334 consensus_queue.cc:260] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.039548 9331 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: No bootstrap required, opened a new log
I20260810 22:58:29.039595 9331 ts_tablet_manager.cc:1397] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:29.039582 9334 ts_tablet_manager.cc:1428] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:29.039599 9344 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.039666 9337 tablet_bootstrap.cc:492] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: No bootstrap required, opened a new log
I20260810 22:58:29.039672 9344 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.039717 9337 ts_tablet_manager.cc:1397] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:29.039753 9331 raft_consensus.cc:348] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.039847 9331 raft_consensus.cc:374] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:29.039868 9331 raft_consensus.cc:729] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9720c8f8630444e9821a69e116c02b5b, State: Initialized, Role: FOLLOWER
I20260810 22:58:29.039866 9337 raft_consensus.cc:348] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.039932 9337 raft_consensus.cc:374] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:29.039952 9337 raft_consensus.cc:729] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed0b5755d294544a3174edb06502790, State: Initialized, Role: FOLLOWER
I20260810 22:58:29.039929 9331 consensus_queue.cc:260] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.040026 9331 ts_tablet_manager.cc:1428] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:29.039994 9337 consensus_queue.cc:260] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.040105 9337 ts_tablet_manager.cc:1428] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:29.040082 9339 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.040174 9339 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.040315 9343 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.040398 9343 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.109916 9344 raft_consensus.cc:482] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:29.110038 9344 raft_consensus.cc:504] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.110260 9344 leader_election.cc:288] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557), 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123)
I20260810 22:58:29.110513 9218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621bc7b7eec2487492ce14e57dd06201" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720c8f8630444e9821a69e116c02b5b" is_pre_election: true
I20260810 22:58:29.110632 9218 raft_consensus.cc:2454] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 0.
I20260810 22:58:29.110559 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621bc7b7eec2487492ce14e57dd06201" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790" is_pre_election: true
I20260810 22:58:29.110682 8681 raft_consensus.cc:2454] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 0.
I20260810 22:58:29.110829 8855 leader_election.cc:302] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed0b5755d294544a3174edb06502790, 552903190a424a15aa41c03667672e49; no voters:
I20260810 22:58:29.110940 9344 raft_consensus.cc:2781] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:29.110986 9344 raft_consensus.cc:482] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:29.111012 9344 raft_consensus.cc:3037] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:29.111825 9344 raft_consensus.cc:504] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.111985 9344 leader_election.cc:288] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 election: Requested vote from peers 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557), 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123)
I20260810 22:58:29.112169 8681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621bc7b7eec2487492ce14e57dd06201" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed0b5755d294544a3174edb06502790"
I20260810 22:58:29.112262 8681 raft_consensus.cc:3037] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:29.112224 9218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621bc7b7eec2487492ce14e57dd06201" candidate_uuid: "552903190a424a15aa41c03667672e49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9720c8f8630444e9821a69e116c02b5b"
I20260810 22:58:29.112315 9218 raft_consensus.cc:3037] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:29.112962 8681 raft_consensus.cc:2454] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 1.
I20260810 22:58:29.113060 9218 raft_consensus.cc:2454] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 552903190a424a15aa41c03667672e49 in term 1.
I20260810 22:58:29.113160 8855 leader_election.cc:302] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ed0b5755d294544a3174edb06502790, 552903190a424a15aa41c03667672e49; no voters:
I20260810 22:58:29.113265 9344 raft_consensus.cc:2781] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:29.113353 9344 raft_consensus.cc:686] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [term 1 LEADER]: Becoming Leader. State: Replica: 552903190a424a15aa41c03667672e49, State: Running, Role: LEADER
I20260810 22:58:29.113416 9344 consensus_queue.cc:237] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } }
I20260810 22:58:29.113725 9347 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: TxnStatusTablet state changed. Reason: New leader 552903190a424a15aa41c03667672e49. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.113813 9347 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:29.113996 9407 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:58:29.114145 9407 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:58:29.114054 8386 catalog_manager.cc:5495] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 552903190a424a15aa41c03667672e49 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:29.134215 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.131352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40527 (local address 127.0.0.1:41408)
0810 22:58:29.131512 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:29.131523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:29.131575 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:29.131733 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:29.131733 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:29.131734 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:29.131781 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.131783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.132712 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.132713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.133319 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.133321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.133811 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.133812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.133856 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.133873 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.134063 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.134070 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 22:58:29.134114 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 22:58:29.134155 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 22:58:29.134215 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.131424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.6.47.130:40527)
0810 22:58:29.131490 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.131492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.131557 (+ 65us) server_negotiation.cc:407] Connection header received
0810 22:58:29.131644 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.131645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.131658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.131720 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:29.131843 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.132627 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.133399 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.133716 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.133805 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.133960 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.133997 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.134055 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.134109 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.134133 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:58:29.135438 9218 raft_consensus.cc:1261] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Refusing update from remote peer 552903190a424a15aa41c03667672e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:29.135474 8681 raft_consensus.cc:1261] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Refusing update from remote peer 552903190a424a15aa41c03667672e49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:29.135705 9344 consensus_queue.cc:1035] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:29.135818 9347 consensus_queue.cc:1035] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:29.136344 9343 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: TxnStatusTablet state changed. Reason: New leader 552903190a424a15aa41c03667672e49. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.136439 9343 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.136426 9339 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: TxnStatusTablet state changed. Reason: New leader 552903190a424a15aa41c03667672e49. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.136535 9339 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.137199 9347 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.137287 9347 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:29.137620 9343 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.137715 9343 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 4ed0b5755d294544a3174edb06502790: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.137959 9344 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.138034 9344 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 552903190a424a15aa41c03667672e49: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:29.138883 9339 tablet_replica.cc:439] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "552903190a424a15aa41c03667672e49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 } } peers { permanent_uuid: "9720c8f8630444e9821a69e116c02b5b" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36557 } } peers { permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 } } }
I20260810 22:58:29.138990 9339 tablet_replica.cc:442] T 621bc7b7eec2487492ce14e57dd06201 P 9720c8f8630444e9821a69e116c02b5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:29.142239 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.139501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.6.47.131:36557)
0810 22:58:29.139558 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.139560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.139569 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:29.139635 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.139635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.139653 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.139712 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:29.139868 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.140455 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.141517 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.141703 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.141755 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.141921 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.142034 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.142102 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.142161 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.142177 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:58:29.147519 9417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.144787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40527 (local address 127.6.47.131:46751)
0810 22:58:29.144965 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:29.144974 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:29.145023 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:29.145142 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:29.145142 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:29.145143 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:29.145213 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.145214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.146112 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.146113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.146690 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.146691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.147144 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.147145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.147191 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.147205 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.147380 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.147381 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:29.147416 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 22:58:29.147460 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":84,"threads_started":1}
I20260810 22:58:29.147518 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.144852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:46751 (local address 127.6.47.130:40527)
0810 22:58:29.144896 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.144903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.145038 (+ 135us) server_negotiation.cc:407] Connection header received
0810 22:58:29.145065 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.145066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.145077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.145134 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:29.145273 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.146033 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.146769 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.147063 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.147133 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.147284 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.147316 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.147370 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.147420 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.147451 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 22:58:29.149951 8681 raft_consensus.cc:1261] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Refusing update from remote peer 9720c8f8630444e9821a69e116c02b5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:29.150122 8980 raft_consensus.cc:1261] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Refusing update from remote peer 9720c8f8630444e9821a69e116c02b5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:29.150218 9339 consensus_queue.cc:1035] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed0b5755d294544a3174edb06502790" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 42123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:29.150379 9386 consensus_queue.cc:1035] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "552903190a424a15aa41c03667672e49" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:29.159487 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.156524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:33387 (local address 127.6.47.131:36557)
0810 22:58:29.156563 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.156565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.156574 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:29.156698 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.156699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.156711 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.156763 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:29.156896 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.157339 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.158632 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.158946 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.158985 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.159228 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.159275 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.159301 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.159403 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.159420 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:29.159690 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.156311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:36557 (local address 127.6.47.130:33387)
0810 22:58:29.156463 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:29.156475 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:29.156563 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:29.156785 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:29.156786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:29.156787 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:29.156833 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.156834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.157406 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.157406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.158443 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:29.158447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.159060 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.159061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:29.159116 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.159131 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:29.159341 (+ 210us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:29.159342 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:29.159569 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 22:58:29.159610 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260810 22:58:29.170332 8386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:29.170439 8386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:29.171145 8386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c9a08829e184192b694632dc3ac7fbd: Sending DeleteTablet for 3 replicas of tablet c408ae6c4f0a4c51a0a0764e40769145
I20260810 22:58:29.171442 8663 tablet_service.cc:1514] Processing DeleteTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:38898
I20260810 22:58:29.171546 9370 tablet_replica.cc:330] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: stopping tablet replica
I20260810 22:58:29.171600 9370 raft_consensus.cc:2229] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:29.171656 9370 raft_consensus.cc:2258] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:29.171908 9186 tablet_service.cc:1514] Processing DeleteTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:36548
I20260810 22:58:29.172004 9367 tablet_replica.cc:330] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: stopping tablet replica
I20260810 22:58:29.172049 9367 raft_consensus.cc:2229] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:29.172140 9367 raft_consensus.cc:2258] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:29.172648 9367 ts_tablet_manager.cc:1905] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:29.174369 9370 ts_tablet_manager.cc:1905] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:29.174490 9367 ts_tablet_manager.cc:1918] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:29.174548 9367 log.cc:1196] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/c408ae6c4f0a4c51a0a0764e40769145
I20260810 22:58:29.174787 9367 ts_tablet_manager.cc:1939] T c408ae6c4f0a4c51a0a0764e40769145 P 9720c8f8630444e9821a69e116c02b5b: Deleting consensus metadata
I20260810 22:58:29.175989 9370 ts_tablet_manager.cc:1918] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:29.176071 9370 log.cc:1196] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/c408ae6c4f0a4c51a0a0764e40769145
I20260810 22:58:29.176288 9370 ts_tablet_manager.cc:1939] T c408ae6c4f0a4c51a0a0764e40769145 P 4ed0b5755d294544a3174edb06502790: Deleting consensus metadata
I20260810 22:58:29.176661 8362 catalog_manager.cc:4841] TS 4ed0b5755d294544a3174edb06502790 (127.6.47.129:42123): tablet c408ae6c4f0a4c51a0a0764e40769145 (table test-table [id=215c2c9828cc43e68d7eea662e75a656]) successfully deleted
I20260810 22:58:29.176753 8362 catalog_manager.cc:4841] TS 9720c8f8630444e9821a69e116c02b5b (127.6.47.131:36557): tablet c408ae6c4f0a4c51a0a0764e40769145 (table test-table [id=215c2c9828cc43e68d7eea662e75a656]) successfully deleted
I20260810 22:58:29.176929 8942 tablet_service.cc:1514] Processing DeleteTablet for tablet c408ae6c4f0a4c51a0a0764e40769145 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:41398
I20260810 22:58:29.177030 9368 tablet_replica.cc:330] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: stopping tablet replica
I20260810 22:58:29.177078 9368 raft_consensus.cc:2229] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:29.177114 9368 raft_consensus.cc:2258] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:29.177443 9368 ts_tablet_manager.cc:1905] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:29.179159 9368 ts_tablet_manager.cc:1918] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:29.179215 9368 log.cc:1196] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/c408ae6c4f0a4c51a0a0764e40769145
I20260810 22:58:29.179443 9368 ts_tablet_manager.cc:1939] T c408ae6c4f0a4c51a0a0764e40769145 P 552903190a424a15aa41c03667672e49: Deleting consensus metadata
I20260810 22:58:29.179860 8362 catalog_manager.cc:4841] TS 552903190a424a15aa41c03667672e49 (127.6.47.130:40527): tablet c408ae6c4f0a4c51a0a0764e40769145 (table test-table [id=215c2c9828cc43e68d7eea662e75a656]) successfully deleted
Aug 10 22:58:29 dist-test-slave-rwrg krb5kdc[8311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402708, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:29.188884 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.177649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.6.47.129:42123)
0810 22:58:29.177701 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.177703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.177990 (+ 287us) server_negotiation.cc:407] Connection header received
0810 22:58:29.178088 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.178088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.178100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.178130 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:29.178911 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.179635 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.180729 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.181034 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.181115 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.185264 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:29.185277 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:29.185277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:29.185286 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:29.187402 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.187842 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.187847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.187847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.187925 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.188161 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.188162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.188162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.188363 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:29.188567 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.188647 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.188743 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260810 22:58:29.189152 8790 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38906
I20260810 22:58:29.198593 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:29.190391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55874 (local address 127.6.47.190:44491)
0810 22:58:29.190449 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:29.190451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:29.190683 (+ 232us) server_negotiation.cc:407] Connection header received
0810 22:58:29.190772 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:29.190773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:29.190786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:29.190840 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:29.191596 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.192383 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.193487 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:29.193812 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:29.193903 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:29.195430 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:29.195431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:29.195432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:29.195439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:29.197260 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.197695 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.197697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.197697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.197758 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:29.198004 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:29.198004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:29.198005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:29.198140 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:29.198241 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:29.198398 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 22:58:29.198489 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 22:58:29.198935 8386 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55874
I20260810 22:58:29.199819 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8527
I20260810 22:58:29.224201 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8823
I20260810 22:58:29.236608 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9070
I20260810 22:58:29.250437 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8328
2026-08-10T22:58:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2839 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:29 dist-test-slave-rwrg krb5kdc[9441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:29 dist-test-slave-rwrg krb5kdc[9441](info): set up 2 sockets
Aug 10 22:58:29 dist-test-slave-rwrg krb5kdc[9441](info): commencing operation
krb5kdc: starting...
W20260810 22:58:30.628697 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:30 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:30.751314 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35005
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:35005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:30.835641 9466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:30.835883 9466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:30.835919 9466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:30.837126 9466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:30.837213 9466 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:58:30.837235 9466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:30.837256 9466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:30.837287 9466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:30.837309 9466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:30.838790 9466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:35005
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:30.839071 9466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:30.839298 9466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:30.844197 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:30.844187 9475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:30.844187 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:30.844522 9466 server_base.cc:1034] running on GCE node
I20260810 22:58:30.844797 9466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:30.845081 9466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:30.846256 9466 hybrid_clock.cc:648] HybridClock initialized: now 1786402710846256 us; error 42 us; skew 500 ppm
Aug 10 22:58:30 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402710, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:30.848749 9466 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:30.849227 9466 webserver.cc:460] Webserver started at http://127.6.47.190:45607/ using document root <none> and password file <none>
I20260810 22:58:30.849472 9466 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:30.849540 9466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:30.849702 9466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:30.850768 9466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "b8a6ef93d040475db79c191d15c6de88"
format_stamp: "Formatted at 2026-08-10 22:58:30 on dist-test-slave-rwrg"
server_key: "17b829679835be294001cb0135c86f84"
server_key_iv: "d25d62fde6145aeee5dbed861d7f4e77"
server_key_version: "encryptionkey@0"
I20260810 22:58:30.851146 9466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "b8a6ef93d040475db79c191d15c6de88"
format_stamp: "Formatted at 2026-08-10 22:58:30 on dist-test-slave-rwrg"
server_key: "17b829679835be294001cb0135c86f84"
server_key_iv: "d25d62fde6145aeee5dbed861d7f4e77"
server_key_version: "encryptionkey@0"
I20260810 22:58:30.852558 9466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:30.853331 9490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:30.853500 9466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:30.853626 9466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "b8a6ef93d040475db79c191d15c6de88"
format_stamp: "Formatted at 2026-08-10 22:58:30 on dist-test-slave-rwrg"
server_key: "17b829679835be294001cb0135c86f84"
server_key_iv: "d25d62fde6145aeee5dbed861d7f4e77"
server_key_version: "encryptionkey@0"
I20260810 22:58:30.853704 9466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:30.869073 9466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:30.870949 9466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:30.871124 9466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:30.876222 9466 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:35005
I20260810 22:58:30.876273 9596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:35005 every 8 connection(s)
I20260810 22:58:30.876696 9466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:30.877377 9598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:30.879690 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Bootstrap starting.
I20260810 22:58:30.880293 9598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:30.880627 9598 log.cc:824] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:30.881326 9598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: No bootstrap required, opened a new log
I20260810 22:58:30.883579 9598 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } }
I20260810 22:58:30.883751 9598 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:30.883805 9598 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a6ef93d040475db79c191d15c6de88, State: Initialized, Role: FOLLOWER
I20260810 22:58:30.883957 9598 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } }
I20260810 22:58:30.884042 9598 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:30.884088 9598 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:30.884140 9598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:30.884797 9598 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } }
I20260810 22:58:30.884938 9598 leader_election.cc:302] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8a6ef93d040475db79c191d15c6de88; no voters:
I20260810 22:58:30.885123 9598 leader_election.cc:288] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:30.885183 9601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:30.885385 9598 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:30.885423 9601 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [term 1 LEADER]: Becoming Leader. State: Replica: b8a6ef93d040475db79c191d15c6de88, State: Running, Role: LEADER
I20260810 22:58:30.885650 9601 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } }
I20260810 22:58:30.886035 9604 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8a6ef93d040475db79c191d15c6de88. Latest consensus state: current_term: 1 leader_uuid: "b8a6ef93d040475db79c191d15c6de88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } } }
I20260810 22:58:30.886118 9604 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:30.886046 9602 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8a6ef93d040475db79c191d15c6de88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a6ef93d040475db79c191d15c6de88" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35005 } } }
I20260810 22:58:30.886346 9602 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:30.886607 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9466
I20260810 22:58:30.886693 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:30.886344 9614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:30.886911 6334 external_mini_cluster.cc:1428] Setting key 3d92034db21f94036a2be12b1fe245ae
I20260810 22:58:30.887338 9614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:30.888935 9614 catalog_manager.cc:1347] Generated new cluster ID: a5d623c28e24458f95d62374a47c3b62
I20260810 22:58:30.889010 9614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:30 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:30.897053 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:30.887385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.6.47.190:35005)
0810 22:58:30.887546 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:30.887554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:30.888070 (+ 516us) server_negotiation.cc:407] Connection header received
0810 22:58:30.888205 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:30.888213 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:30.888267 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:30.888428 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:30.889177 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:30.890120 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:30.890886 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:30.891306 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:30.891424 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:30.893835 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:30.893839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:30.893840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:30.893852 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:30.895523 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:30.895880 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:30.895883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:30.895884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:30.895984 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:30.896204 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:30.896205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:30.896205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:30.896429 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:30.896630 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:30.896709 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 22:58:30.896854 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260810 22:58:30.907970 9614 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:30.908682 9614 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:30.915843 9614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Generated new TSK 0
I20260810 22:58:30.916018 9614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:30.947443 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35005
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:31.030566 9642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:31.030812 9642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:31.030854 9642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:31.032012 9642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:31.032083 9642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:31.032125 9642 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:58:31.032155 9642 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:31.032186 9642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:31.033704 9642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:35005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:31.033999 9642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:31.034250 9642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:31.039388 9657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.039407 9653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.039388 9652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:31.039572 9642 server_base.cc:1034] running on GCE node
I20260810 22:58:31.039944 9642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:31.040213 9642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:31.041379 9642 hybrid_clock.cc:648] HybridClock initialized: now 1786402711041364 us; error 45 us; skew 500 ppm
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:31.043993 9642 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:31.044456 9642 webserver.cc:460] Webserver started at http://127.6.47.129:41457/ using document root <none> and password file <none>
I20260810 22:58:31.044740 9642 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:31.044816 9642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:31.044981 9642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:31.046051 9642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "dd732d0dd5a74d658b18f120a2afadad"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "4ed07b289cb13a3167fd67bd91397d61"
server_key_iv: "1320f23af89bcd18c3cc4a7035ea4e09"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.046429 9642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "dd732d0dd5a74d658b18f120a2afadad"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "4ed07b289cb13a3167fd67bd91397d61"
server_key_iv: "1320f23af89bcd18c3cc4a7035ea4e09"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.047755 9642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:31.048481 9668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.048695 9642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:31.048800 9642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "dd732d0dd5a74d658b18f120a2afadad"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "4ed07b289cb13a3167fd67bd91397d61"
server_key_iv: "1320f23af89bcd18c3cc4a7035ea4e09"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.048887 9642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:31.075713 9642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:31.077643 9642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:31.077827 9642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:31.078418 9642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:31.078488 9642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.078541 9642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:31.078567 9642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:31.086374 9642 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:37649
I20260810 22:58:31.086539 9874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:37649 every 8 connection(s)
I20260810 22:58:31.086855 9642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:31.089990 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.079018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:43393 (local address 127.6.47.190:35005)
0810 22:58:31.079096 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.079099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.079375 (+ 276us) server_negotiation.cc:407] Connection header received
0810 22:58:31.080127 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.080128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.080141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.080182 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.081162 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.081968 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.083675 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.084042 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.084134 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.086768 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.086770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.086770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.086778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.088454 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.088936 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.088937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.088937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.089000 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.089292 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.089293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.089293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.089420 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.089494 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.089785 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.089896 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:58:31.090128 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.078986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.129:43393)
0810 22:58:31.079224 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.079276 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.080008 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.080287 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.080291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.080735 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.081055 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.081063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.082070 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.082072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.083574 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.083578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.084147 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.084148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.084298 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.084853 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.084862 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.086678 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.088598 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.088605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.088608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.088852 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.089085 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.089086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.089086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.089208 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.089497 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.089505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.089643 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.089747 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.089907 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":122,"threads_started":1}
I20260810 22:58:31.090677 9879 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35005
I20260810 22:58:31.090766 9879 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:31.090956 9879 heartbeater.cc:511] Master 127.6.47.190:35005 requested a full tablet report, sending...
I20260810 22:58:31.091504 9515 ts_manager.cc:194] Registered new tserver with Master: dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649)
I20260810 22:58:31.092453 9515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:43393
I20260810 22:58:31.093194 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9642
I20260810 22:58:31.093318 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:31.093513 6334 external_mini_cluster.cc:1428] Setting key 64fa5102b69b101b4dd74d97bb13574b
I20260810 22:58:31.097774 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.091763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55675 (local address 127.6.47.190:35005)
0810 22:58:31.091804 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.091806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.091881 (+ 75us) server_negotiation.cc:407] Connection header received
0810 22:58:31.091960 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.091960 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.091967 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.092014 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.092337 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.093036 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.093910 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.094112 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.094188 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.095128 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.095130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.095130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.095138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.096655 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.097078 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.097078 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.097078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.097133 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.097350 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.097350 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.097350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.097442 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.097492 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.097652 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.097717 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:31.097896 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.091662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.129:55675)
0810 22:58:31.091815 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.091826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.091903 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.092020 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.092021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.092215 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.092274 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.092275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.093138 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.093139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.093834 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.093836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.094187 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.094188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.094266 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.094713 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.094718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.095015 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.096742 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.096743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.096743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.096984 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.097200 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.097201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.097201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.097276 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.097507 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.097508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.097558 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.097638 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.097750 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":19,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:31.119072 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35005
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:31.202095 9896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:31.202306 9896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:31.202328 9896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:31.203446 9896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:31.203498 9896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:31.203521 9896 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:58:31.203532 9896 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:31.203548 9896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:31.205037 9896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:35005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:31.205232 9896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:31.205410 9896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:31.210402 9909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.210400 9903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.210446 9904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:31.210834 9896 server_base.cc:1034] running on GCE node
I20260810 22:58:31.211037 9896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:31.211287 9896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:31.212450 9896 hybrid_clock.cc:648] HybridClock initialized: now 1786402711212410 us; error 56 us; skew 500 ppm
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:31.214823 9896 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:31.215242 9896 webserver.cc:460] Webserver started at http://127.6.47.130:42723/ using document root <none> and password file <none>
I20260810 22:58:31.215458 9896 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:31.215523 9896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:31.215679 9896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:31.216805 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "5d03546a68244586aa20297441ff857d"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "c1fcfa59483b2332bda55db74e440bb7"
server_key_iv: "943b8d1ac6ac1b22e4beb780f52bd865"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.217163 9896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "5d03546a68244586aa20297441ff857d"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "c1fcfa59483b2332bda55db74e440bb7"
server_key_iv: "943b8d1ac6ac1b22e4beb780f52bd865"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.218973 9896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 22:58:31.219746 9925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.219928 9896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:31.220041 9896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "5d03546a68244586aa20297441ff857d"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "c1fcfa59483b2332bda55db74e440bb7"
server_key_iv: "943b8d1ac6ac1b22e4beb780f52bd865"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.220146 9896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:31.234867 9896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:31.236860 9896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:31.237013 9896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:31.237743 9896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:31.237798 9896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.237828 9896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:31.237845 9896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:31.249143 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.238119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.130:60529)
0810 22:58:31.238393 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.238440 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.239263 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.239611 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.239616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.239972 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.240258 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.240265 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.241248 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.241250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.242645 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.242648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.243203 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.243204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.243373 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.243989 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.243998 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.245957 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.247663 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.247669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.247671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.247937 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.248110 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.248110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.248110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.248221 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.248469 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.248473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.248642 (+ 169us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.248730 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.248883 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":88,"threads_started":1}
I20260810 22:58:31.249887 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.238124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:60529 (local address 127.6.47.190:35005)
0810 22:58:31.238174 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.238177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.238529 (+ 352us) server_negotiation.cc:407] Connection header received
0810 22:58:31.239412 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.239413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.239430 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.239518 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.240369 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.241129 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.242750 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.243053 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.243118 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.246053 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.246054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.246054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.246062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.247573 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.248003 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.248004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.248004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.248055 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.248276 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.248277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.248277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.248399 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.248449 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.249593 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.249740 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:31.251922 9896 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:43633
I20260810 22:58:31.252005 10138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:43633 every 8 connection(s)
I20260810 22:58:31.252311 9896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:31.254591 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9896
I20260810 22:58:31.254699 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:31.254899 6334 external_mini_cluster.cc:1428] Setting key ebd6d07362110918978f779d646e219d
I20260810 22:58:31.253193 10141 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35005
I20260810 22:58:31.255409 10141 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:31.255614 10141 heartbeater.cc:511] Master 127.6.47.190:35005 requested a full tablet report, sending...
I20260810 22:58:31.256036 9515 ts_manager.cc:194] Registered new tserver with Master: 5d03546a68244586aa20297441ff857d (127.6.47.130:43633)
I20260810 22:58:31.256541 9515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:60529
I20260810 22:58:31.259749 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.251997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.130:40029)
0810 22:58:31.252667 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.252680 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.252737 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.253007 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.253008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.253252 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.253300 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.253302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.254001 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.254002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.254804 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.254807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.255243 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.255244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.255320 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.256647 (+ 1327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.256653 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.257015 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.258632 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.258634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.258634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.258890 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.259103 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.259108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.259108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.259163 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.259505 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.259506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.259542 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.259595 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.259679 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":605,"threads_started":1}
I20260810 22:58:31.259864 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.252773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:40029 (local address 127.6.47.190:35005)
0810 22:58:31.252881 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.252883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.252891 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:31.252908 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.252909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.252916 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.252999 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.253387 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.253901 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.254883 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.255169 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.255244 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.257109 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.257110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.257110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.257118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.258550 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.258970 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.258971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.258972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.259033 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.259229 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.259231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.259231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.259434 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.259509 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.259647 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.259732 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:31.282943 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35005
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:31.369340 10147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:31.369542 10147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:31.369567 10147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:31.370682 10147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:31.370719 10147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:31.370739 10147 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 22:58:31.370751 10147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:31.370769 10147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:31.372275 10147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:35005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:31.372471 10147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:31.372730 10147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:31.377779 10156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.377768 10155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:31.377830 10158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:31.378012 10147 server_base.cc:1034] running on GCE node
I20260810 22:58:31.378217 10147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:31.378459 10147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:31.379602 10147 hybrid_clock.cc:648] HybridClock initialized: now 1786402711379575 us; error 40 us; skew 500 ppm
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:31.382164 10147 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:31.382606 10147 webserver.cc:460] Webserver started at http://127.6.47.131:41943/ using document root <none> and password file <none>
I20260810 22:58:31.382853 10147 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:31.382897 10147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:31.383062 10147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:31.384109 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "3abfce7fa69a4b2f899abede120c9f20"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "2e171814fd8ab8ad5870fa97442d8b7f"
server_key_iv: "4c0c3f441220fda3d89bfea3abde5ece"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.384485 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "3abfce7fa69a4b2f899abede120c9f20"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "2e171814fd8ab8ad5870fa97442d8b7f"
server_key_iv: "4c0c3f441220fda3d89bfea3abde5ece"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.386154 10147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:31.387071 10172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.387272 10147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:31.387372 10147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "3abfce7fa69a4b2f899abede120c9f20"
format_stamp: "Formatted at 2026-08-10 22:58:31 on dist-test-slave-rwrg"
server_key: "2e171814fd8ab8ad5870fa97442d8b7f"
server_key_iv: "4c0c3f441220fda3d89bfea3abde5ece"
server_key_version: "encryptionkey@0"
I20260810 22:58:31.387456 10147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:31.404335 10147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:31.406224 10147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:31.406406 10147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:31.407066 10147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:31.407140 10147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.407187 10147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:31.407209 10147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.414770 10147 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:46793
I20260810 22:58:31.414865 10382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:46793 every 8 connection(s)
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:31.415179 10147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:31.418443 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.407536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:35433 (local address 127.6.47.190:35005)
0810 22:58:31.407613 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.407616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.407837 (+ 221us) server_negotiation.cc:407] Connection header received
0810 22:58:31.408384 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.408385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.408400 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.408513 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.409646 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.410503 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.412237 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.412595 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.412666 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.415543 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.415544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.415544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.415553 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.417034 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.417445 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.417447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.417447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.417503 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.417789 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.417801 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.417802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.417919 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.417990 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.418274 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.418356 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:58:31.418641 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.407521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.131:35433)
0810 22:58:31.407706 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.407746 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.408222 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.408621 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.408627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.409176 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.409492 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.409499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.410625 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.410627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.412118 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.412122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.412696 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.412698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.412885 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.413542 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.413555 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.415423 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.417138 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.417145 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.417148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.417367 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.417584 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.417585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.417586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.417720 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.417993 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.418000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.418138 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.418238 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.418392 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":117,"threads_started":1}
I20260810 22:58:31.418640 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10147
I20260810 22:58:31.418723 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:31.418898 6334 external_mini_cluster.cc:1428] Setting key 043d323ed7a09287725ad0bd6e07a155
I20260810 22:58:31.419345 10384 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35005
I20260810 22:58:31.419430 10384 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:31.419611 10384 heartbeater.cc:511] Master 127.6.47.190:35005 requested a full tablet report, sending...
I20260810 22:58:31.420003 9512 ts_manager.cc:194] Registered new tserver with Master: 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:31.420475 9512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:35433
I20260810 22:58:31.425729 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.420541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:54221 (local address 127.6.47.190:35005)
0810 22:58:31.420580 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.420582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.420710 (+ 128us) server_negotiation.cc:407] Connection header received
0810 22:58:31.420766 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.420767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.420778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.420828 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.421159 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.421626 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.422338 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.422512 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.422573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.423428 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.423429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.423430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.423435 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.424943 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.425217 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.425217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.425218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.425258 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.425403 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.425404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.425404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.425480 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.425523 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.425624 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.425675 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:31.425750 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.420468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.6.47.131:54221)
0810 22:58:31.420640 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.420651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.420727 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.420836 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.420836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.421029 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.421086 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.421088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.421746 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.421747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.422266 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.422267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.422586 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.422587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.422640 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.423023 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.423063 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.423357 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.425009 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.425010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.425010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.425160 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.425308 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.425309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.425309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.425360 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.425526 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.425527 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.425560 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.425609 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.425690 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260810 22:58:31.430565 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:31.454412 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.447450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58164 (local address 127.6.47.190:35005)
0810 22:58:31.447520 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.447522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.447530 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:31.447595 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.447595 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.447605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.447658 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.448193 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.448726 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.449429 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.449609 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.449658 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.451913 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.451915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.451915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.451922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.453469 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.453776 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.453777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.453777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.453828 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.453990 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.453991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.453991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.454092 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.454178 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.454261 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.454320 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:58:31.456029 9512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:31.457818 9512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:31.464596 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37649 (local address 127.0.0.1:42972)
0810 22:58:31.460820 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.460827 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.460888 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.461163 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.461164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.461166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.461210 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.461211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.462552 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.462553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.463506 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.463509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.464225 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.464226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.464310 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.464324 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.464431 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.464510 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260810 22:58:31.464608 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.6.47.129:37649)
0810 22:58:31.460654 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.460659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.460948 (+ 289us) server_negotiation.cc:407] Connection header received
0810 22:58:31.460998 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.461000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.461017 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.461090 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.461318 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.462449 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.463597 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.464119 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.464221 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.464286 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.464449 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.464515 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260810 22:58:31.464740 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46793 (local address 127.0.0.1:50934)
0810 22:58:31.460392 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.460433 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.460539 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.460965 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.460969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.460972 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.461049 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.461053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.462474 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.462475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.463359 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.463380 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.464391 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.464392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.464473 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.464477 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.464580 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.464626 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260810 22:58:31.465029 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.6.47.131:46793)
0810 22:58:31.460707 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.460710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.460719 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:31.460776 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.460778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.460796 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.460877 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.461136 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.462309 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.463480 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.464232 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.464304 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.464356 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.464880 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.464935 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":85,"threads_started":1}
I20260810 22:58:31.465428 9759 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 (DEFAULT_TABLE table=test-table [id=b26b394ed167429da5dd787227c2db29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.465616 10277 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 (DEFAULT_TABLE table=test-table [id=b26b394ed167429da5dd787227c2db29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.465823 9759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5067d51ca14951ab4bc1dae077e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.466029 10277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5067d51ca14951ab4bc1dae077e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.466343 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.6.47.130:43633)
0810 22:58:31.460621 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.460626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.460770 (+ 144us) server_negotiation.cc:407] Connection header received
0810 22:58:31.460838 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.460840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.460856 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.460925 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.461135 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.462297 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.464905 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.465676 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.465753 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.465807 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.466179 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.466246 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
I20260810 22:58:31.466511 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.460205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.0.0.1:56392)
0810 22:58:31.460683 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.460693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.460761 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.460992 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.460993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.460995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.461049 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.461051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.463501 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.463502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.464593 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.464595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.465768 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.465769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.465952 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.465957 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.466017 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.466052 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260810 22:58:31.467203 10005 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 (DEFAULT_TABLE table=test-table [id=b26b394ed167429da5dd787227c2db29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.467562 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5067d51ca14951ab4bc1dae077e669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.469367 10429 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Bootstrap starting.
I20260810 22:58:31.469671 10431 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Bootstrap starting.
I20260810 22:58:31.470413 10429 tablet_bootstrap.cc:654] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.470413 10431 tablet_bootstrap.cc:654] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.470791 10429 log.cc:824] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:31.470784 10431 log.cc:824] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:31.471181 10432 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Bootstrap starting.
I20260810 22:58:31.471657 10431 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: No bootstrap required, opened a new log
I20260810 22:58:31.471778 10431 ts_tablet_manager.cc:1397] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:31.471807 10432 tablet_bootstrap.cc:654] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.471913 10429 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: No bootstrap required, opened a new log
I20260810 22:58:31.471987 10429 ts_tablet_manager.cc:1397] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:31.472095 10432 log.cc:824] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:31.472908 10432 tablet_bootstrap.cc:492] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: No bootstrap required, opened a new log
I20260810 22:58:31.472992 10432 ts_tablet_manager.cc:1397] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:31.474064 10429 raft_consensus.cc:348] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474076 10431 raft_consensus.cc:348] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474236 10431 raft_consensus.cc:374] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.474236 10429 raft_consensus.cc:374] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.474272 10431 raft_consensus.cc:729] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.474272 10429 raft_consensus.cc:729] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.474399 10429 consensus_queue.cc:260] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474432 10431 consensus_queue.cc:260] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474360 10432 raft_consensus.cc:348] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474545 10432 raft_consensus.cc:374] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.474586 10432 raft_consensus.cc:729] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d03546a68244586aa20297441ff857d, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.474726 10429 ts_tablet_manager.cc:1428] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 22:58:31.474726 10431 ts_tablet_manager.cc:1428] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:31.474706 10432 consensus_queue.cc:260] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.474866 9879 heartbeater.cc:503] Master 127.6.47.190:35005 was elected leader, sending a full tablet report...
I20260810 22:58:31.474866 10384 heartbeater.cc:503] Master 127.6.47.190:35005 was elected leader, sending a full tablet report...
I20260810 22:58:31.475286 10432 ts_tablet_manager.cc:1428] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:31.475432 10141 heartbeater.cc:503] Master 127.6.47.190:35005 was elected leader, sending a full tablet report...
I20260810 22:58:31.479671 10440 raft_consensus.cc:482] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:31.479748 10440 raft_consensus.cc:504] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.479972 10440 leader_election.cc:288] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d03546a68244586aa20297441ff857d (127.6.47.130:43633), dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649)
I20260810 22:58:31.482904 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.480154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37649 (local address 127.6.47.131:38445)
0810 22:58:31.480218 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.480229 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.480270 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.480401 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.480402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.480403 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.480484 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.480485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.481123 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.481124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.482040 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.482043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.482651 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.482738 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.482741 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.482805 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.482842 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 22:58:31.482903 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.480204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:38445 (local address 127.6.47.129:37649)
0810 22:58:31.480253 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.480255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.480262 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:31.480327 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.480327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.480337 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.480396 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.480583 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.481034 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482124 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.482558 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482651 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.482660 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.482811 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.482842 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:31.483062 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.480253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:47181 (local address 127.6.47.130:43633)
0810 22:58:31.480309 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.480311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.480537 (+ 226us) server_negotiation.cc:407] Connection header received
0810 22:58:31.480624 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.480625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.480635 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.480689 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.480833 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.481284 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482382 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.482723 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482802 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.482809 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.482972 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.483003 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:58:31.483075 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.480153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.6.47.131:47181)
0810 22:58:31.480449 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.480460 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.480567 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.480692 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.480692 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.480694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.480757 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.480758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.481374 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.481375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.482263 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.482266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.482802 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.482803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.482883 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.482892 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.482954 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.482992 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":123,"thread_start_us":94,"threads_started":1}
I20260810 22:58:31.483188 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5067d51ca14951ab4bc1dae077e669" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad" is_pre_election: true
I20260810 22:58:31.483304 9794 raft_consensus.cc:2454] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 0.
I20260810 22:58:31.483505 10174 leader_election.cc:302] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abfce7fa69a4b2f899abede120c9f20, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:31.483729 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5067d51ca14951ab4bc1dae077e669" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d" is_pre_election: true
I20260810 22:58:31.483825 10050 raft_consensus.cc:2454] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 0.
I20260810 22:58:31.483904 10440 raft_consensus.cc:2781] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:31.483985 10440 raft_consensus.cc:482] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:31.484011 10440 raft_consensus.cc:3037] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.484688 10440 raft_consensus.cc:504] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.484825 10440 leader_election.cc:288] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 election: Requested vote from peers 5d03546a68244586aa20297441ff857d (127.6.47.130:43633), dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649)
I20260810 22:58:31.484958 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5067d51ca14951ab4bc1dae077e669" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d"
I20260810 22:58:31.484989 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5067d51ca14951ab4bc1dae077e669" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad"
I20260810 22:58:31.485037 10050 raft_consensus.cc:3037] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.485046 9794 raft_consensus.cc:3037] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.485862 10050 raft_consensus.cc:2454] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 1.
I20260810 22:58:31.485904 9794 raft_consensus.cc:2454] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 1.
I20260810 22:58:31.486084 10173 leader_election.cc:302] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abfce7fa69a4b2f899abede120c9f20, 5d03546a68244586aa20297441ff857d; no voters:
I20260810 22:58:31.486191 10440 raft_consensus.cc:2781] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:31.486342 10440 raft_consensus.cc:686] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 LEADER]: Becoming Leader. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Running, Role: LEADER
I20260810 22:58:31.486451 10440 consensus_queue.cc:237] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } }
I20260810 22:58:31.487399 9515 catalog_manager.cc:5495] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "3abfce7fa69a4b2f899abede120c9f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } health_report { overall_health: UNKNOWN } } }
W20260810 22:58:31.503041 10142 tablet.cc:2369] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 22:58:31 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402710, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:31.524622 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.517148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35005 (local address 127.0.0.1:58168)
0810 22:58:31.517344 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.517355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.517447 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.517597 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.517598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.517932 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:31.517975 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.517977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.518628 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.518629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.519411 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.519415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.519810 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.519811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.519865 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.520460 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:31.520464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:31.521835 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:31.523491 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.523499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.523501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.523715 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.523919 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:31.523920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:31.523920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:31.523982 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:31.524222 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:31.524229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:31.524274 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.524359 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.524472 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260810 22:58:31.524623 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.517231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.6.47.190:35005)
0810 22:58:31.517319 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.517322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.517432 (+ 110us) server_negotiation.cc:407] Connection header received
0810 22:58:31.517500 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.517501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.517512 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.517593 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:31.518071 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.518517 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.519501 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.519703 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.519810 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.521939 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:31.521940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:31.521940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:31.521948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:31.523393 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.523790 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.523791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.523792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.523846 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:31.524046 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:31.524047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:31.524047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:31.524159 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:31.524230 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.524383 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.524547 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":59}
I20260810 22:58:31.526767 9515 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:58168:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:58:31.526999 9515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:31.528554 10005 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc843c4f1d4411db1897309fb4a4ab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0c0012d2feb455194eee4c0c7de9430]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:31.528760 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc843c4f1d4411db1897309fb4a4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.528788 10277 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc843c4f1d4411db1897309fb4a4ab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0c0012d2feb455194eee4c0c7de9430]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:31.528952 10277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc843c4f1d4411db1897309fb4a4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.530174 10432 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: Bootstrap starting.
I20260810 22:58:31.530373 10429 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: Bootstrap starting.
I20260810 22:58:31.530511 9759 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc843c4f1d4411db1897309fb4a4ab6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a0c0012d2feb455194eee4c0c7de9430]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:31.530674 9759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc843c4f1d4411db1897309fb4a4ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.530844 10429 tablet_bootstrap.cc:654] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.530905 10432 tablet_bootstrap.cc:654] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.531560 10429 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: No bootstrap required, opened a new log
I20260810 22:58:31.531637 10429 ts_tablet_manager.cc:1397] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:31.531803 10429 raft_consensus.cc:348] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.531874 10429 raft_consensus.cc:374] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.531894 10429 raft_consensus.cc:729] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.531939 10429 consensus_queue.cc:260] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.532052 10429 ts_tablet_manager.cc:1428] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.532215 10440 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.532300 10440 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.532354 10431 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: Bootstrap starting.
I20260810 22:58:31.533102 10431 tablet_bootstrap.cc:654] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.533655 10432 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: No bootstrap required, opened a new log
I20260810 22:58:31.533716 10432 ts_tablet_manager.cc:1397] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 22:58:31.533843 10431 tablet_bootstrap.cc:492] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: No bootstrap required, opened a new log
I20260810 22:58:31.533861 10432 raft_consensus.cc:348] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.533906 10431 ts_tablet_manager.cc:1397] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:31.533929 10432 raft_consensus.cc:374] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.533949 10432 raft_consensus.cc:729] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d03546a68244586aa20297441ff857d, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.534001 10432 consensus_queue.cc:260] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.534062 10431 raft_consensus.cc:348] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.534102 10432 ts_tablet_manager.cc:1428] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.534130 10431 raft_consensus.cc:374] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.534152 10431 raft_consensus.cc:729] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.534158 10441 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.534190 10431 consensus_queue.cc:260] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.534246 10441 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.534281 10431 ts_tablet_manager.cc:1428] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:31.534457 10439 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.534555 10439 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 22:58:31.587970 9880 tablet.cc:2369] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:31.607415 10441 raft_consensus.cc:482] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:31.607499 10441 raft_consensus.cc:504] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.607827 10441 leader_election.cc:288] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:31.610699 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.608077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:52501 (local address 127.6.47.129:37649)
0810 22:58:31.608146 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.608147 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.608157 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:31.608185 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.608186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.608195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.608249 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.608473 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.608987 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.609967 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.610319 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.610421 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.610431 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.610590 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.610632 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:31.610699 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.608004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37649 (local address 127.6.47.130:52501)
0810 22:58:31.608044 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.608055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.608125 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.608267 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.608268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.608270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.608359 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.608362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.609078 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.609079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.609870 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.609872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.610435 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.610437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.610513 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.610516 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.610582 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.610626 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260810 22:58:31.611052 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.608004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46793 (local address 127.6.47.130:57163)
0810 22:58:31.608347 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.608356 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.608426 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.608595 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.608595 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.608597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:31.608660 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.608661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.609311 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.609312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.610040 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.610042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.610615 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.610616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.610691 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.610694 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.610927 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.610969 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":173,"thread_start_us":97,"threads_started":1}
I20260810 22:58:31.611084 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.608077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:57163 (local address 127.6.47.131:46793)
0810 22:58:31.608134 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.608136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.608433 (+ 297us) server_negotiation.cc:407] Connection header received
0810 22:58:31.608510 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.608511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.608524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.608591 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:31.608767 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.609191 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.610132 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.610508 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.610594 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.610603 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.610963 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.611015 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:58:31.611033 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc843c4f1d4411db1897309fb4a4ab6" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad" is_pre_election: true
I20260810 22:58:31.611135 9794 raft_consensus.cc:2454] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 0.
I20260810 22:58:31.611348 9929 leader_election.cc:302] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:31.611428 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc843c4f1d4411db1897309fb4a4ab6" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20" is_pre_election: true
I20260810 22:58:31.611495 10441 raft_consensus.cc:2781] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:31.611542 10441 raft_consensus.cc:482] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:31.611549 10305 raft_consensus.cc:2454] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 0.
I20260810 22:58:31.611584 10441 raft_consensus.cc:3037] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.612334 10441 raft_consensus.cc:504] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.612511 10441 leader_election.cc:288] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 election: Requested vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:31.612670 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc843c4f1d4411db1897309fb4a4ab6" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad"
I20260810 22:58:31.612703 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc843c4f1d4411db1897309fb4a4ab6" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20"
I20260810 22:58:31.612744 9794 raft_consensus.cc:3037] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.612780 10305 raft_consensus.cc:3037] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.613333 9794 raft_consensus.cc:2454] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 1.
I20260810 22:58:31.613494 10305 raft_consensus.cc:2454] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 1.
I20260810 22:58:31.613556 9929 leader_election.cc:302] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:31.613642 10441 raft_consensus.cc:2781] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:31.613799 10441 raft_consensus.cc:686] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [term 1 LEADER]: Becoming Leader. State: Replica: 5d03546a68244586aa20297441ff857d, State: Running, Role: LEADER
I20260810 22:58:31.613902 10441 consensus_queue.cc:237] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.614369 10475 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: TxnStatusTablet state changed. Reason: New leader 5d03546a68244586aa20297441ff857d. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.614461 10475 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:31.614732 9512 catalog_manager.cc:5495] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d03546a68244586aa20297441ff857d (127.6.47.130). New cstate: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:31.614991 10477 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:58:31.615147 10477 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:58:31.627074 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.624573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56408 (local address 127.6.47.130:43633)
0810 22:58:31.624617 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.624619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.624627 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:31.624663 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.624664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.624679 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.624735 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:31.624902 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.625359 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.626349 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.626524 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.626573 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.626779 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.626865 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.626922 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.626996 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.627010 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 22:58:31.627134 10464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.624328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.0.0.1:56408)
0810 22:58:31.624416 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.624429 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.624479 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.624766 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.624767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.624769 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:31.624820 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.624823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.625427 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.625428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.626268 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.626271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.626600 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.626601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.626669 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.626686 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.626930 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.626938 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.626991 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.627037 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 22:58:31.628834 10305 raft_consensus.cc:1261] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Refusing update from remote peer 5d03546a68244586aa20297441ff857d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.628834 9794 raft_consensus.cc:1261] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Refusing update from remote peer 5d03546a68244586aa20297441ff857d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.629123 10441 consensus_queue.cc:1035] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:31.629261 10475 consensus_queue.cc:1035] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:31.630499 10439 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: TxnStatusTablet state changed. Reason: New leader 5d03546a68244586aa20297441ff857d. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.630597 10439 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.630677 10440 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: TxnStatusTablet state changed. Reason: New leader 5d03546a68244586aa20297441ff857d. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.630779 10440 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.631546 10441 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.631637 10441 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:31.631709 10440 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.631796 10440 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 3abfce7fa69a4b2f899abede120c9f20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.632303 10480 mvcc.cc:205] Tried to move back new op lower bound from 7317105506829148160 to 7317105506771128320. Current Snapshot: MvccSnapshot[applied={T|T < 7317105506829148160}]
I20260810 22:58:31.632617 10482 mvcc.cc:205] Tried to move back new op lower bound from 7317105506829148160 to 7317105506771128320. Current Snapshot: MvccSnapshot[applied={T|T < 7317105506829148160}]
I20260810 22:58:31.633414 10475 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.633500 10475 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P 5d03546a68244586aa20297441ff857d: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:31.634013 10439 tablet_replica.cc:439] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } } }
I20260810 22:58:31.634092 10439 tablet_replica.cc:442] T 3cc843c4f1d4411db1897309fb4a4ab6 P dd732d0dd5a74d658b18f120a2afadad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:31.634245 10481 mvcc.cc:205] Tried to move back new op lower bound from 7317105506829148160 to 7317105506771128320. Current Snapshot: MvccSnapshot[applied={T|T < 7317105506829148160}]
I20260810 22:58:31.639349 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.636880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.6.47.131:46793)
0810 22:58:31.636925 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.636927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.636936 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:31.637026 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.637027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.637045 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.637111 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:31.637274 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.637746 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.638529 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.638742 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.638819 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.639010 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.639115 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.639209 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.639249 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.639273 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 22:58:31.644301 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.641961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:43009 (local address 127.6.47.130:43633)
0810 22:58:31.642012 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.642014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.642022 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:31.642044 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.642045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.642057 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.642101 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:31.642246 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.642652 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.643703 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.643865 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.643913 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.644124 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.644147 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.644183 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.644241 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.644255 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 22:58:31.644348 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.641875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.6.47.131:43009)
0810 22:58:31.641942 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.641953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.642002 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.642123 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.642124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.642125 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:31.642185 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.642187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.642720 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.642721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.643616 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.643618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.643936 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.643937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.644009 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.644025 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.644190 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.644191 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.644239 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.644279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 22:58:31.649483 10050 raft_consensus.cc:1261] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Refusing update from remote peer 3abfce7fa69a4b2f899abede120c9f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.649681 9794 raft_consensus.cc:1261] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Refusing update from remote peer 3abfce7fa69a4b2f899abede120c9f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.649770 10443 consensus_queue.cc:1035] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:31.649899 10440 consensus_queue.cc:1035] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:31.659403 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.656043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:41611 (local address 127.6.47.131:46793)
0810 22:58:31.656100 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:31.656102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:31.656168 (+ 66us) server_negotiation.cc:407] Connection header received
0810 22:58:31.656221 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:31.656221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:31.656229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:31.656274 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:31.656432 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.656905 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.658773 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.658957 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.659050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.659228 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.659251 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.659285 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:31.659343 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 22:58:31.659356 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:58:31.659453 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:31.656049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46793 (local address 127.6.47.130:41611)
0810 22:58:31.656117 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:31.656126 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:31.656177 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:31.656299 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:31.656300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:31.656301 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:31.656365 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.656366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.656991 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.656992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.658680 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:31.658683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:31.659059 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:31.659060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:31.659124 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:31.659141 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:31.659291 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:31.659293 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 22:58:31.659341 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:58:31.659382 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 22:58:31.672863 9512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:31.672979 9512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:31.673718 9512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Sending DeleteTablet for 3 replicas of tablet 4a5067d51ca14951ab4bc1dae077e669
I20260810 22:58:31.674027 10277 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:31 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 22:58:31.674027 9759 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:31 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42972
I20260810 22:58:31.674510 10005 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5067d51ca14951ab4bc1dae077e669 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:31 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:56392
I20260810 22:58:31.674724 9512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:31.674889 9512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:31.675280 10513 tablet_replica.cc:330] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: stopping tablet replica
I20260810 22:58:31.676159 9759 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4fafc1e774091b174427048e6d29e (DEFAULT_TABLE table=test-table [id=40b66caa1c304ae384d9536c9694c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.676302 9759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4fafc1e774091b174427048e6d29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.676481 10005 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4fafc1e774091b174427048e6d29e (DEFAULT_TABLE table=test-table [id=40b66caa1c304ae384d9536c9694c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.676620 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4fafc1e774091b174427048e6d29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.677999 10431 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Bootstrap starting.
I20260810 22:58:31.678508 10431 tablet_bootstrap.cc:654] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.678917 10432 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Bootstrap starting.
I20260810 22:58:31.679354 10432 tablet_bootstrap.cc:654] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.679379 10431 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: No bootstrap required, opened a new log
I20260810 22:58:31.679419 10431 ts_tablet_manager.cc:1397] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:31.679580 10431 raft_consensus.cc:348] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.679656 10431 raft_consensus.cc:374] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.679680 10431 raft_consensus.cc:729] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.679737 10431 consensus_queue.cc:260] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.679836 10431 ts_tablet_manager.cc:1428] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.676177 10277 tablet_service.cc:1467] Processing CreateTablet for tablet 55d4fafc1e774091b174427048e6d29e (DEFAULT_TABLE table=test-table [id=40b66caa1c304ae384d9536c9694c1f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:31.680709 10513 raft_consensus.cc:2229] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:31.680729 10277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d4fafc1e774091b174427048e6d29e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:31.680832 10513 raft_consensus.cc:2258] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:31.681005 10520 tablet_replica.cc:330] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: stopping tablet replica
I20260810 22:58:31.681097 10520 raft_consensus.cc:2229] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:31.681155 10520 raft_consensus.cc:2258] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:31.681279 10516 tablet_replica.cc:330] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: stopping tablet replica
I20260810 22:58:31.681371 10516 raft_consensus.cc:2229] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:31.681419 10516 raft_consensus.cc:2258] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:31.681450 10513 ts_tablet_manager.cc:1905] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:31.681851 10516 ts_tablet_manager.cc:1905] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:31.682230 10429 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Bootstrap starting.
I20260810 22:58:31.682631 10429 tablet_bootstrap.cc:654] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:31.683499 10516 ts_tablet_manager.cc:1918] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:31.683580 10516 log.cc:1196] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/4a5067d51ca14951ab4bc1dae077e669
I20260810 22:58:31.683890 10516 ts_tablet_manager.cc:1939] T 4a5067d51ca14951ab4bc1dae077e669 P dd732d0dd5a74d658b18f120a2afadad: Deleting consensus metadata
I20260810 22:58:31.683991 10429 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: No bootstrap required, opened a new log
I20260810 22:58:31.684034 10429 ts_tablet_manager.cc:1397] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:58:31.684172 10429 raft_consensus.cc:348] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.684242 10429 raft_consensus.cc:374] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.684268 10429 raft_consensus.cc:729] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.684312 10429 consensus_queue.cc:260] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.684324 9492 catalog_manager.cc:4841] TS dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649): tablet 4a5067d51ca14951ab4bc1dae077e669 (table test-table [id=b26b394ed167429da5dd787227c2db29]) successfully deleted
I20260810 22:58:31.684423 10429 ts_tablet_manager.cc:1428] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:31.684855 10513 ts_tablet_manager.cc:1918] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:31.684926 10513 log.cc:1196] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/4a5067d51ca14951ab4bc1dae077e669
I20260810 22:58:31.685173 10513 ts_tablet_manager.cc:1939] T 4a5067d51ca14951ab4bc1dae077e669 P 3abfce7fa69a4b2f899abede120c9f20: Deleting consensus metadata
I20260810 22:58:31.685576 9495 catalog_manager.cc:4841] TS 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793): tablet 4a5067d51ca14951ab4bc1dae077e669 (table test-table [id=b26b394ed167429da5dd787227c2db29]) successfully deleted
I20260810 22:58:31.685698 10520 ts_tablet_manager.cc:1905] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:31.687194 10520 ts_tablet_manager.cc:1918] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:31.687268 10520 log.cc:1196] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/4a5067d51ca14951ab4bc1dae077e669
I20260810 22:58:31.687500 10520 ts_tablet_manager.cc:1939] T 4a5067d51ca14951ab4bc1dae077e669 P 5d03546a68244586aa20297441ff857d: Deleting consensus metadata
I20260810 22:58:31.687879 9491 catalog_manager.cc:4841] TS 5d03546a68244586aa20297441ff857d (127.6.47.130:43633): tablet 4a5067d51ca14951ab4bc1dae077e669 (table test-table [id=b26b394ed167429da5dd787227c2db29]) successfully deleted
I20260810 22:58:31.688148 10432 tablet_bootstrap.cc:492] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: No bootstrap required, opened a new log
I20260810 22:58:31.688215 10432 ts_tablet_manager.cc:1397] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260810 22:58:31.688357 10432 raft_consensus.cc:348] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.688427 10432 raft_consensus.cc:374] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:31.688455 10432 raft_consensus.cc:729] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d03546a68244586aa20297441ff857d, State: Initialized, Role: FOLLOWER
I20260810 22:58:31.688498 10432 consensus_queue.cc:260] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.688616 10432 ts_tablet_manager.cc:1428] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260810 22:58:31.916481 10385 tablet.cc:2369] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:31.952644 10443 raft_consensus.cc:482] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:31.952765 10443 raft_consensus.cc:504] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.952991 10443 leader_election.cc:288] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 5d03546a68244586aa20297441ff857d (127.6.47.130:43633)
I20260810 22:58:31.953246 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4fafc1e774091b174427048e6d29e" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d" is_pre_election: true
I20260810 22:58:31.953349 10050 raft_consensus.cc:2454] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 0.
I20260810 22:58:31.953277 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4fafc1e774091b174427048e6d29e" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad" is_pre_election: true
I20260810 22:58:31.953392 9794 raft_consensus.cc:2454] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 0.
I20260810 22:58:31.953537 10173 leader_election.cc:302] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abfce7fa69a4b2f899abede120c9f20, 5d03546a68244586aa20297441ff857d; no voters:
I20260810 22:58:31.953641 10443 raft_consensus.cc:2781] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:31.953711 10443 raft_consensus.cc:482] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:31.953728 10443 raft_consensus.cc:3037] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.954476 10443 raft_consensus.cc:504] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.954624 10443 leader_election.cc:288] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 election: Requested vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 5d03546a68244586aa20297441ff857d (127.6.47.130:43633)
I20260810 22:58:31.954907 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4fafc1e774091b174427048e6d29e" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad"
I20260810 22:58:31.954902 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d4fafc1e774091b174427048e6d29e" candidate_uuid: "3abfce7fa69a4b2f899abede120c9f20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d"
I20260810 22:58:31.954978 10050 raft_consensus.cc:3037] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.955000 9794 raft_consensus.cc:3037] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:31.955600 10050 raft_consensus.cc:2454] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 1.
I20260810 22:58:31.955690 9794 raft_consensus.cc:2454] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abfce7fa69a4b2f899abede120c9f20 in term 1.
I20260810 22:58:31.955801 10173 leader_election.cc:302] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abfce7fa69a4b2f899abede120c9f20, 5d03546a68244586aa20297441ff857d; no voters:
I20260810 22:58:31.955909 10443 raft_consensus.cc:2781] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:31.955971 10443 raft_consensus.cc:686] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 1 LEADER]: Becoming Leader. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Running, Role: LEADER
I20260810 22:58:31.956023 10443 consensus_queue.cc:237] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:31.956741 9515 catalog_manager.cc:5495] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "3abfce7fa69a4b2f899abede120c9f20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } health_report { overall_health: HEALTHY } } }
I20260810 22:58:31.969274 9794 raft_consensus.cc:1261] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Refusing update from remote peer 3abfce7fa69a4b2f899abede120c9f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.969259 10050 raft_consensus.cc:1261] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Refusing update from remote peer 3abfce7fa69a4b2f899abede120c9f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:31.969470 10443 consensus_queue.cc:1035] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:31.969568 10440 consensus_queue.cc:1035] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:58:31.974345 10482 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 22:58:33.492274 9515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:33.492403 9515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:33.493275 9515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Sending DeleteTablet for 3 replicas of tablet 55d4fafc1e774091b174427048e6d29e
I20260810 22:58:33.493528 9759 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4fafc1e774091b174427048e6d29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42972
I20260810 22:58:33.493561 10277 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4fafc1e774091b174427048e6d29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 22:58:33.493769 10005 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d4fafc1e774091b174427048e6d29e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:56392
I20260810 22:58:33.493970 10572 tablet_replica.cc:330] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: stopping tablet replica
I20260810 22:58:33.493968 10573 tablet_replica.cc:330] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: stopping tablet replica
I20260810 22:58:33.494025 10573 raft_consensus.cc:2229] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:33.494071 10572 raft_consensus.cc:2229] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:33.494114 10573 raft_consensus.cc:2258] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.494141 10572 raft_consensus.cc:2258] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.494381 10573 ts_tablet_manager.cc:1905] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.494403 10572 ts_tablet_manager.cc:1905] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.496011 10573 ts_tablet_manager.cc:1918] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:58:33.496145 10573 log.cc:1196] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/55d4fafc1e774091b174427048e6d29e
I20260810 22:58:33.496428 10572 ts_tablet_manager.cc:1918] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:58:33.496460 10573 ts_tablet_manager.cc:1939] T 55d4fafc1e774091b174427048e6d29e P 5d03546a68244586aa20297441ff857d: Deleting consensus metadata
I20260810 22:58:33.496598 10572 log.cc:1196] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/55d4fafc1e774091b174427048e6d29e
I20260810 22:58:33.496905 10572 ts_tablet_manager.cc:1939] T 55d4fafc1e774091b174427048e6d29e P dd732d0dd5a74d658b18f120a2afadad: Deleting consensus metadata
I20260810 22:58:33.496891 9491 catalog_manager.cc:4841] TS 5d03546a68244586aa20297441ff857d (127.6.47.130:43633): tablet 55d4fafc1e774091b174427048e6d29e (table test-table [id=40b66caa1c304ae384d9536c9694c1f0]) successfully deleted
I20260810 22:58:33.497269 9492 catalog_manager.cc:4841] TS dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649): tablet 55d4fafc1e774091b174427048e6d29e (table test-table [id=40b66caa1c304ae384d9536c9694c1f0]) successfully deleted
I20260810 22:58:33.498936 10571 tablet_replica.cc:330] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: stopping tablet replica
I20260810 22:58:33.498996 10571 raft_consensus.cc:2229] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:33.499085 10571 raft_consensus.cc:2258] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.499353 10571 ts_tablet_manager.cc:1905] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.500640 10571 ts_tablet_manager.cc:1918] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:58:33.500723 10571 log.cc:1196] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/55d4fafc1e774091b174427048e6d29e
I20260810 22:58:33.500962 10571 ts_tablet_manager.cc:1939] T 55d4fafc1e774091b174427048e6d29e P 3abfce7fa69a4b2f899abede120c9f20: Deleting consensus metadata
I20260810 22:58:33.501283 9495 catalog_manager.cc:4841] TS 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793): tablet 55d4fafc1e774091b174427048e6d29e (table test-table [id=40b66caa1c304ae384d9536c9694c1f0]) successfully deleted
Aug 10 22:58:33 dist-test-slave-rwrg krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:58:33 dist-test-slave-rwrg krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:33.528437 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:33.512072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58174 (local address 127.6.47.190:35005)
0810 22:58:33.512237 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:33.512239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:33.512249 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:33.512275 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:33.512275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:33.512285 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:33.512348 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:33.512945 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.513445 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.514154 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.514349 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.514398 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:33.526231 (+ 11833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:33.526232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:33.526232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:33.526239 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:33.527487 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:33.527779 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:33.527780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:33.527780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:33.527828 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:33.527981 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:33.527981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:33.527981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:33.528079 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:33.528149 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:33.528254 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 22:58:33.528319 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260810 22:58:33.530046 9515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:33.530191 9515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:33.531582 10005 tablet_service.cc:1467] Processing CreateTablet for tablet 6280636425b34f468cf489993b82ebf1 (DEFAULT_TABLE table=test-table [id=6d00355235d94701826d048981dc8c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.531627 10277 tablet_service.cc:1467] Processing CreateTablet for tablet 6280636425b34f468cf489993b82ebf1 (DEFAULT_TABLE table=test-table [id=6d00355235d94701826d048981dc8c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.531757 10277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6280636425b34f468cf489993b82ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.531759 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6280636425b34f468cf489993b82ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.532603 9759 tablet_service.cc:1467] Processing CreateTablet for tablet 6280636425b34f468cf489993b82ebf1 (DEFAULT_TABLE table=test-table [id=6d00355235d94701826d048981dc8c87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.532759 9759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6280636425b34f468cf489993b82ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.533183 10588 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Bootstrap starting.
I20260810 22:58:33.533318 10589 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Bootstrap starting.
I20260810 22:58:33.533566 10588 tablet_bootstrap.cc:654] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.533664 10589 tablet_bootstrap.cc:654] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.534229 10588 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: No bootstrap required, opened a new log
I20260810 22:58:33.534238 10594 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Bootstrap starting.
I20260810 22:58:33.534291 10588 ts_tablet_manager.cc:1397] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:33.534430 10588 raft_consensus.cc:348] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.534497 10588 raft_consensus.cc:374] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.534511 10588 raft_consensus.cc:729] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.534566 10588 consensus_queue.cc:260] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.534616 10594 tablet_bootstrap.cc:654] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.534672 10588 ts_tablet_manager.cc:1428] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:33.534914 10589 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: No bootstrap required, opened a new log
I20260810 22:58:33.534971 10589 ts_tablet_manager.cc:1397] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:33.535100 10589 raft_consensus.cc:348] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.535198 10589 raft_consensus.cc:374] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.535223 10589 raft_consensus.cc:729] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d03546a68244586aa20297441ff857d, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.535189 10594 tablet_bootstrap.cc:492] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: No bootstrap required, opened a new log
I20260810 22:58:33.535256 10594 ts_tablet_manager.cc:1397] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:33.535280 10589 consensus_queue.cc:260] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.535398 10589 ts_tablet_manager.cc:1428] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:33.535398 10594 raft_consensus.cc:348] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.535460 10594 raft_consensus.cc:374] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.535480 10594 raft_consensus.cc:729] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.535537 10594 consensus_queue.cc:260] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.535697 10594 ts_tablet_manager.cc:1428] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:33.817364 10546 raft_consensus.cc:482] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:33.817463 10546 raft_consensus.cc:504] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.817662 10546 leader_election.cc:288] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:33.817901 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6280636425b34f468cf489993b82ebf1" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad" is_pre_election: true
I20260810 22:58:33.817909 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6280636425b34f468cf489993b82ebf1" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20" is_pre_election: true
I20260810 22:58:33.818009 9794 raft_consensus.cc:2454] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 0.
I20260810 22:58:33.818011 10305 raft_consensus.cc:2454] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 0.
I20260810 22:58:33.818233 9939 leader_election.cc:302] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3abfce7fa69a4b2f899abede120c9f20, 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:33.818348 10546 raft_consensus.cc:2781] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:33.818403 10546 raft_consensus.cc:482] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:33.818434 10546 raft_consensus.cc:3037] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:33.819252 10546 raft_consensus.cc:504] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.819408 10546 leader_election.cc:288] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 election: Requested vote from peers dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:33.819609 9794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6280636425b34f468cf489993b82ebf1" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd732d0dd5a74d658b18f120a2afadad"
I20260810 22:58:33.819595 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6280636425b34f468cf489993b82ebf1" candidate_uuid: "5d03546a68244586aa20297441ff857d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20"
I20260810 22:58:33.819685 9794 raft_consensus.cc:3037] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:33.819705 10305 raft_consensus.cc:3037] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:33.820439 10305 raft_consensus.cc:2454] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 1.
I20260810 22:58:33.820438 9794 raft_consensus.cc:2454] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d03546a68244586aa20297441ff857d in term 1.
I20260810 22:58:33.820652 9929 leader_election.cc:302] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:33.820766 10546 raft_consensus.cc:2781] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:33.820822 10546 raft_consensus.cc:686] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 1 LEADER]: Becoming Leader. State: Replica: 5d03546a68244586aa20297441ff857d, State: Running, Role: LEADER
I20260810 22:58:33.820874 10546 consensus_queue.cc:237] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.821563 9515 catalog_manager.cc:5495] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d03546a68244586aa20297441ff857d (127.6.47.130). New cstate: current_term: 1 leader_uuid: "5d03546a68244586aa20297441ff857d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:33.892702 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:33.889507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.6.47.130:43633)
0810 22:58:33.889668 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:33.889670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:33.889678 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:33.889703 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:33.889703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:33.889715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:33.889791 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:33.889951 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.890852 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.891929 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.892185 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.892248 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:33.892391 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:33.892430 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:33.892480 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:33.892564 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 22:58:33.892601 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":116,"threads_started":1}
I20260810 22:58:33.897964 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:33.895306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:52843 (local address 127.6.47.130:43633)
0810 22:58:33.895351 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:33.895353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:33.895361 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:33.895422 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:33.895422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:33.895431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:33.895474 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:33.895586 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.896032 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.897267 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.897530 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.897593 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:33.897709 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:33.897738 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:33.897788 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:33.897842 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 22:58:33.897880 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":19}
I20260810 22:58:33.897965 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:33.895144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.6.47.130:52843)
0810 22:58:33.895298 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:33.895309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:33.895364 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:33.895480 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:33.895481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:33.895483 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:33.895532 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:33.895533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.896112 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.896113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:33.897149 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:33.897152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:33.897598 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:33.897599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:33.897647 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:33.897650 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:33.897789 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:33.897791 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 22:58:33.897839 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:58:33.897896 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260810 22:58:33.900442 9794 raft_consensus.cc:1261] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Refusing update from remote peer 5d03546a68244586aa20297441ff857d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:33.900467 10305 raft_consensus.cc:1261] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Refusing update from remote peer 5d03546a68244586aa20297441ff857d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:33.900702 10499 consensus_queue.cc:1035] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:33.900848 10499 consensus_queue.cc:1035] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:33.911625 9515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:33.911761 9515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:33.912416 9515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Sending DeleteTablet for 3 replicas of tablet 6280636425b34f468cf489993b82ebf1
I20260810 22:58:33.912756 9759 tablet_service.cc:1514] Processing DeleteTablet for tablet 6280636425b34f468cf489993b82ebf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42972
I20260810 22:58:33.912808 10005 tablet_service.cc:1514] Processing DeleteTablet for tablet 6280636425b34f468cf489993b82ebf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:56392
I20260810 22:58:33.912866 10277 tablet_service.cc:1514] Processing DeleteTablet for tablet 6280636425b34f468cf489993b82ebf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:33 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 22:58:33.912904 10573 tablet_replica.cc:330] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: stopping tablet replica
I20260810 22:58:33.912951 10572 tablet_replica.cc:330] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: stopping tablet replica
I20260810 22:58:33.912966 10573 raft_consensus.cc:2229] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:33.912998 10572 raft_consensus.cc:2229] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:33.913038 10572 raft_consensus.cc:2258] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.913051 10573 raft_consensus.cc:2258] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.913117 10571 tablet_replica.cc:330] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: stopping tablet replica
I20260810 22:58:33.913169 10571 raft_consensus.cc:2229] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:33.913201 10571 raft_consensus.cc:2258] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:33.913341 10573 ts_tablet_manager.cc:1905] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.913599 10572 ts_tablet_manager.cc:1905] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.913563 9515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:33.913714 9515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:33.915311 10573 ts_tablet_manager.cc:1918] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:33.915337 10572 ts_tablet_manager.cc:1918] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:33.915373 10573 log.cc:1196] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/6280636425b34f468cf489993b82ebf1
I20260810 22:58:33.915385 10572 log.cc:1196] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/6280636425b34f468cf489993b82ebf1
I20260810 22:58:33.915478 10571 ts_tablet_manager.cc:1905] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:33.915606 10573 ts_tablet_manager.cc:1939] T 6280636425b34f468cf489993b82ebf1 P 5d03546a68244586aa20297441ff857d: Deleting consensus metadata
I20260810 22:58:33.915606 10572 ts_tablet_manager.cc:1939] T 6280636425b34f468cf489993b82ebf1 P dd732d0dd5a74d658b18f120a2afadad: Deleting consensus metadata
I20260810 22:58:33.915913 9491 catalog_manager.cc:4841] TS 5d03546a68244586aa20297441ff857d (127.6.47.130:43633): tablet 6280636425b34f468cf489993b82ebf1 (table test-table [id=6d00355235d94701826d048981dc8c87]) successfully deleted
I20260810 22:58:33.916666 9492 catalog_manager.cc:4841] TS dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649): tablet 6280636425b34f468cf489993b82ebf1 (table test-table [id=6d00355235d94701826d048981dc8c87]) successfully deleted
I20260810 22:58:33.917299 10571 ts_tablet_manager.cc:1918] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:33.917366 10571 log.cc:1196] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/6280636425b34f468cf489993b82ebf1
I20260810 22:58:33.917593 10571 ts_tablet_manager.cc:1939] T 6280636425b34f468cf489993b82ebf1 P 3abfce7fa69a4b2f899abede120c9f20: Deleting consensus metadata
I20260810 22:58:33.917909 9495 catalog_manager.cc:4841] TS 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793): tablet 6280636425b34f468cf489993b82ebf1 (table test-table [id=6d00355235d94701826d048981dc8c87]) successfully deleted
I20260810 22:58:33.918088 9759 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b (DEFAULT_TABLE table=test-table [id=3d9489e973ca4dc5bb4a87048f8eae75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.918216 9759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ba8e8a654cffbd33f736ef93c71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.918218 10005 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b (DEFAULT_TABLE table=test-table [id=3d9489e973ca4dc5bb4a87048f8eae75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.918334 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ba8e8a654cffbd33f736ef93c71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.919525 10594 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: Bootstrap starting.
I20260810 22:58:33.919536 10589 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Bootstrap starting.
I20260810 22:58:33.919870 10589 tablet_bootstrap.cc:654] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.919922 10594 tablet_bootstrap.cc:654] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.920176 10277 tablet_service.cc:1467] Processing CreateTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b (DEFAULT_TABLE table=test-table [id=3d9489e973ca4dc5bb4a87048f8eae75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:33.920271 10277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9198ba8e8a654cffbd33f736ef93c71b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:33.920850 10594 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: No bootstrap required, opened a new log
I20260810 22:58:33.920917 10594 ts_tablet_manager.cc:1397] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:33.921072 10594 raft_consensus.cc:348] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.921150 10594 raft_consensus.cc:374] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.921175 10594 raft_consensus.cc:729] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.921226 10594 consensus_queue.cc:260] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.921322 10594 ts_tablet_manager.cc:1428] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:33.921450 10588 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Bootstrap starting.
I20260810 22:58:33.921825 10588 tablet_bootstrap.cc:654] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:33.922354 10589 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: No bootstrap required, opened a new log
I20260810 22:58:33.922410 10589 ts_tablet_manager.cc:1397] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 22:58:33.922508 10589 raft_consensus.cc:348] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.922577 10589 raft_consensus.cc:374] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.922596 10589 raft_consensus.cc:729] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d03546a68244586aa20297441ff857d, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.922637 10589 consensus_queue.cc:260] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.922725 10589 ts_tablet_manager.cc:1428] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:33.923017 10588 tablet_bootstrap.cc:492] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: No bootstrap required, opened a new log
I20260810 22:58:33.923071 10588 ts_tablet_manager.cc:1397] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:33.923177 10588 raft_consensus.cc:348] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.923241 10588 raft_consensus.cc:374] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:33.923269 10588 raft_consensus.cc:729] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abfce7fa69a4b2f899abede120c9f20, State: Initialized, Role: FOLLOWER
I20260810 22:58:33.923317 10588 consensus_queue.cc:260] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:33.923406 10588 ts_tablet_manager.cc:1428] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:34.076171 10596 raft_consensus.cc:482] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:34.076287 10596 raft_consensus.cc:504] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:34.076763 10596 leader_election.cc:288] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d03546a68244586aa20297441ff857d (127.6.47.130:43633), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:34.080830 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.077035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:57049 (local address 127.6.47.130:43633)
0810 22:58:34.077091 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:34.077093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:34.077233 (+ 140us) server_negotiation.cc:407] Connection header received
0810 22:58:34.077271 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:34.077272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:34.077282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:34.077350 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:34.077567 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.078299 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.079884 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.080422 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.080545 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.080555 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:34.080719 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 22:58:34.080757 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 22:58:34.080832 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.077069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:46979 (local address 127.6.47.131:46793)
0810 22:58:34.077189 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:34.077191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:34.077239 (+ 48us) server_negotiation.cc:407] Connection header received
0810 22:58:34.077271 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:34.077272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:34.077283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:34.077352 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:34.077564 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.078302 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.079884 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.080431 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.080545 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.080555 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:34.080708 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 22:58:34.080755 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260810 22:58:34.080870 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.076964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46793 (local address 127.6.47.129:46979)
0810 22:58:34.077093 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:34.077104 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:34.077159 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:34.077381 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:34.077382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:34.077386 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:34.077478 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.077480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.078392 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.078393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.079791 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.079794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.080545 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.080545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.080619 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.080624 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:34.080684 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 22:58:34.080733 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":113,"thread_start_us":64,"threads_started":1}
I20260810 22:58:34.080874 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.076970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.6.47.129:57049)
0810 22:58:34.077154 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:34.077163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:34.077219 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:34.077381 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:34.077382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:34.077384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:34.077478 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.077480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.078416 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.078417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.079791 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.079794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.080543 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.080544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.080622 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.080625 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:34.080691 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:58:34.080733 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":65,"thread_start_us":122,"threads_started":1}
I20260810 22:58:34.081188 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ba8e8a654cffbd33f736ef93c71b" candidate_uuid: "dd732d0dd5a74d658b18f120a2afadad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d" is_pre_election: true
I20260810 22:58:34.081282 10050 raft_consensus.cc:2454] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd732d0dd5a74d658b18f120a2afadad in term 0.
I20260810 22:58:34.081493 9670 leader_election.cc:302] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:34.081501 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ba8e8a654cffbd33f736ef93c71b" candidate_uuid: "dd732d0dd5a74d658b18f120a2afadad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20" is_pre_election: true
I20260810 22:58:34.081594 10304 raft_consensus.cc:2454] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd732d0dd5a74d658b18f120a2afadad in term 0.
I20260810 22:58:34.081681 10596 raft_consensus.cc:2781] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:34.081745 10596 raft_consensus.cc:482] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:34.081780 10596 raft_consensus.cc:3037] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:34.082471 10596 raft_consensus.cc:504] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:34.082633 10596 leader_election.cc:288] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [CANDIDATE]: Term 1 election: Requested vote from peers 5d03546a68244586aa20297441ff857d (127.6.47.130:43633), 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793)
I20260810 22:58:34.082809 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ba8e8a654cffbd33f736ef93c71b" candidate_uuid: "dd732d0dd5a74d658b18f120a2afadad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d03546a68244586aa20297441ff857d"
I20260810 22:58:34.082809 10304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9198ba8e8a654cffbd33f736ef93c71b" candidate_uuid: "dd732d0dd5a74d658b18f120a2afadad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abfce7fa69a4b2f899abede120c9f20"
I20260810 22:58:34.082891 10304 raft_consensus.cc:3037] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:34.082891 10050 raft_consensus.cc:3037] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:34.083657 10050 raft_consensus.cc:2454] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd732d0dd5a74d658b18f120a2afadad in term 1.
I20260810 22:58:34.083657 10304 raft_consensus.cc:2454] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd732d0dd5a74d658b18f120a2afadad in term 1.
I20260810 22:58:34.083853 9670 leader_election.cc:302] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d03546a68244586aa20297441ff857d, dd732d0dd5a74d658b18f120a2afadad; no voters:
I20260810 22:58:34.083981 10596 raft_consensus.cc:2781] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:34.084151 10596 raft_consensus.cc:686] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 1 LEADER]: Becoming Leader. State: Replica: dd732d0dd5a74d658b18f120a2afadad, State: Running, Role: LEADER
I20260810 22:58:34.084260 10596 consensus_queue.cc:237] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } }
I20260810 22:58:34.085073 9515 catalog_manager.cc:5495] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad reported cstate change: term changed from 0 to 1, leader changed from <none> to dd732d0dd5a74d658b18f120a2afadad (127.6.47.129). New cstate: current_term: 1 leader_uuid: "dd732d0dd5a74d658b18f120a2afadad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd732d0dd5a74d658b18f120a2afadad" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:34.102545 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.100274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.6.47.129:37649)
0810 22:58:34.100454 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:34.100456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:34.100465 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:34.100527 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:34.100527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:34.100545 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:34.100620 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:34.100704 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.101169 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.101830 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.102020 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.102070 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.102209 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:34.102313 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:34.102369 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:34.102410 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 22:58:34.102428 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260810 22:58:34.107051 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.104999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:46347 (local address 127.6.47.130:43633)
0810 22:58:34.105045 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:34.105047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:34.105078 (+ 31us) server_negotiation.cc:407] Connection header received
0810 22:58:34.105132 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:34.105132 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:34.105143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:34.105190 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:34.105320 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.105764 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.106457 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.106650 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.106702 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.106868 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:34.106892 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:34.106930 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:34.106982 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 22:58:34.106999 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:34.107075 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.104849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43633 (local address 127.6.47.129:46347)
0810 22:58:34.105019 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:34.105030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:34.105091 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:34.105207 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:34.105207 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:34.105209 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:34.105263 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.105264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.105824 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.105825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.106383 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.106385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.106738 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.106739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.106783 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.106798 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:34.106945 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:34.106947 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 22:58:34.106981 (+ 34us) client_negotiation.cc:240] Negotiation successful
0810 22:58:34.107012 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260810 22:58:34.109426 10050 raft_consensus.cc:1261] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Refusing update from remote peer dd732d0dd5a74d658b18f120a2afadad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:34.109454 10304 raft_consensus.cc:1261] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Refusing update from remote peer dd732d0dd5a74d658b18f120a2afadad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:34.109687 10596 consensus_queue.cc:1035] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abfce7fa69a4b2f899abede120c9f20" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:34.109946 10596 consensus_queue.cc:1035] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d03546a68244586aa20297441ff857d" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:34.117496 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.115149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:54949 (local address 127.6.47.129:37649)
0810 22:58:34.115184 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:34.115186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:34.115218 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:58:34.115307 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:34.115308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:34.115319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:34.115383 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:34.115478 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.115966 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.116814 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.117084 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.117159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.117317 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:34.117351 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:34.117388 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:34.117435 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 22:58:34.117452 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:58:34.117517 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:34.115122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37649 (local address 127.6.47.130:54949)
0810 22:58:34.115165 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:34.115182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:34.115233 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:34.115387 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:34.115389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:34.115390 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:34.115428 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.115430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.116049 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.116050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.116728 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:34.116731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:34.117173 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:34.117174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:34.117237 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:34.117244 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:34.117390 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:34.117392 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 22:58:34.117434 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 22:58:34.117470 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
W20260810 22:58:34.118000 10542 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 22:58:35.642673 9511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:35.642776 9511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:35.643579 9511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8a6ef93d040475db79c191d15c6de88: Sending DeleteTablet for 3 replicas of tablet 9198ba8e8a654cffbd33f736ef93c71b
I20260810 22:58:35.643874 9759 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:35 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42972
I20260810 22:58:35.643908 9995 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:35 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:56392
I20260810 22:58:35.643930 10277 tablet_service.cc:1514] Processing DeleteTablet for tablet 9198ba8e8a654cffbd33f736ef93c71b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:35 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:50934
I20260810 22:58:35.644097 10674 tablet_replica.cc:330] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad: stopping tablet replica
I20260810 22:58:35.644161 10674 raft_consensus.cc:2229] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:35.644227 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9642
I20260810 22:58:35.644268 10674 raft_consensus.cc:2258] T 9198ba8e8a654cffbd33f736ef93c71b P dd732d0dd5a74d658b18f120a2afadad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:35.653862 10679 tablet_replica.cc:330] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: stopping tablet replica
I20260810 22:58:35.653862 10677 tablet_replica.cc:330] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: stopping tablet replica
I20260810 22:58:35.653945 10677 raft_consensus.cc:2229] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:35.653990 10677 raft_consensus.cc:2258] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:35.654063 10679 raft_consensus.cc:2229] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:35.654147 10679 raft_consensus.cc:2258] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:35.654790 10679 ts_tablet_manager.cc:1905] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:35.654960 10677 ts_tablet_manager.cc:1905] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:35.656867 10677 ts_tablet_manager.cc:1918] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:58:35.656955 10677 log.cc:1196] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/9198ba8e8a654cffbd33f736ef93c71b
I20260810 22:58:35.657037 10679 ts_tablet_manager.cc:1918] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 22:58:35.657094 10679 log.cc:1196] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/9198ba8e8a654cffbd33f736ef93c71b
I20260810 22:58:35.657196 10677 ts_tablet_manager.cc:1939] T 9198ba8e8a654cffbd33f736ef93c71b P 3abfce7fa69a4b2f899abede120c9f20: Deleting consensus metadata
I20260810 22:58:35.657330 10679 ts_tablet_manager.cc:1939] T 9198ba8e8a654cffbd33f736ef93c71b P 5d03546a68244586aa20297441ff857d: Deleting consensus metadata
I20260810 22:58:35.657652 9491 catalog_manager.cc:4841] TS 5d03546a68244586aa20297441ff857d (127.6.47.130:43633): tablet 9198ba8e8a654cffbd33f736ef93c71b (table test-table [id=3d9489e973ca4dc5bb4a87048f8eae75]) successfully deleted
I20260810 22:58:35.658111 9495 catalog_manager.cc:4841] TS 3abfce7fa69a4b2f899abede120c9f20 (127.6.47.131:46793): tablet 9198ba8e8a654cffbd33f736ef93c71b (table test-table [id=3d9489e973ca4dc5bb4a87048f8eae75]) successfully deleted
W20260810 22:58:35.661353 9492 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:37649 (error 108)
I20260810 22:58:35.661952 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9896
I20260810 22:58:35.662158 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:35.661867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37649 (local address 127.0.0.1:60358)
0810 22:58:35.662051 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:35.662113 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
W20260810 22:58:35.669003 9492 catalog_manager.cc:4584] TS dd732d0dd5a74d658b18f120a2afadad (127.6.47.129:37649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9198ba8e8a654cffbd33f736ef93c71b: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37649: connect: Connection refused (error 111)
I20260810 22:58:35.675807 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10147
I20260810 22:58:35.686933 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9466
2026-08-10T22:58:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6439 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:35 dist-test-slave-rwrg krb5kdc[10689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:35 dist-test-slave-rwrg krb5kdc[10689](info): set up 2 sockets
Aug 10 22:58:35 dist-test-slave-rwrg krb5kdc[10689](info): commencing operation
krb5kdc: starting...
W20260810 22:58:37.034935 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:37.157248 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:43923
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:43923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:37.240272 10708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:37.240568 10708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:37.240617 10708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:37.241778 10708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:37.241822 10708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:37.241837 10708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:37.241850 10708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:37.241902 10708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:37.243377 10708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:43923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:43923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:37.243599 10708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:37.243831 10708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:37.249190 10717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.249341 10708 server_base.cc:1034] running on GCE node
W20260810 22:58:37.249191 10714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.249365 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.249864 10708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:37.250178 10708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:37.251353 10708 hybrid_clock.cc:648] HybridClock initialized: now 1786402717251333 us; error 39 us; skew 500 ppm
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:37.253948 10708 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:37.254441 10708 webserver.cc:460] Webserver started at http://127.6.47.190:38163/ using document root <none> and password file <none>
I20260810 22:58:37.254673 10708 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:37.254742 10708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:37.254927 10708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:37.255980 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "640bb2ba46464896b30d48406ac663a2"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "02b9a17d61bda204593abe0c12e7079d"
server_key_iv: "2942ad5841221aae38efed3a07c3b0de"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.256373 10708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "640bb2ba46464896b30d48406ac663a2"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "02b9a17d61bda204593abe0c12e7079d"
server_key_iv: "2942ad5841221aae38efed3a07c3b0de"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.257726 10708 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:37.258453 10734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.258627 10708 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:37.258736 10708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "640bb2ba46464896b30d48406ac663a2"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "02b9a17d61bda204593abe0c12e7079d"
server_key_iv: "2942ad5841221aae38efed3a07c3b0de"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.258837 10708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:37.283697 10708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:37.285670 10708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:37.285844 10708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:37.290416 10708 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:43923
I20260810 22:58:37.290462 10844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:43923 every 8 connection(s)
I20260810 22:58:37.290868 10708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:37.291781 10845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:37.292434 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10708
I20260810 22:58:37.292567 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:37.292780 6334 external_mini_cluster.cc:1428] Setting key 28938b574b97882e7310942638cd2db7
I20260810 22:58:37.295235 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2: Bootstrap starting.
I20260810 22:58:37.296025 10845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:37.296357 10845 log.cc:824] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:37.297257 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2: No bootstrap required, opened a new log
I20260810 22:58:37.299392 10845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } }
I20260810 22:58:37.299569 10845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:37.299613 10845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640bb2ba46464896b30d48406ac663a2, State: Initialized, Role: FOLLOWER
I20260810 22:58:37.299765 10845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } }
I20260810 22:58:37.299847 10845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:37.299876 10845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:37.299930 10845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:37.300670 10845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } }
I20260810 22:58:37.300791 10845 leader_election.cc:302] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 640bb2ba46464896b30d48406ac663a2; no voters:
I20260810 22:58:37.301023 10845 leader_election.cc:288] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:37.301080 10850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:37.301312 10845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:37.301317 10850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [term 1 LEADER]: Becoming Leader. State: Replica: 640bb2ba46464896b30d48406ac663a2, State: Running, Role: LEADER
I20260810 22:58:37.301481 10850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } }
I20260810 22:58:37.301826 10854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 640bb2ba46464896b30d48406ac663a2. Latest consensus state: current_term: 1 leader_uuid: "640bb2ba46464896b30d48406ac663a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } } }
I20260810 22:58:37.301842 10851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "640bb2ba46464896b30d48406ac663a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640bb2ba46464896b30d48406ac663a2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43923 } } }
I20260810 22:58:37.302028 10851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:37.302263 10854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:37.302363 10874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:37.302994 10874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:37.304530 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.293379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.6.47.190:43923)
0810 22:58:37.293594 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:37.293605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:37.293644 (+ 39us) server_negotiation.cc:407] Connection header received
0810 22:58:37.294206 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:37.294215 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:37.294278 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:37.294450 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:37.295035 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.296394 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.297258 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.297846 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.297970 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.300384 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:37.300389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:37.300389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:37.300402 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:37.302751 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.303177 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.303181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.303182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.303298 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.303522 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.303523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.303524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.303836 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:37.303965 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:37.304152 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 22:58:37.304285 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
I20260810 22:58:37.304790 10874 catalog_manager.cc:1347] Generated new cluster ID: 30153bdffc334f1b97f23f4a3e7b256b
I20260810 22:58:37.304839 10874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:37.327975 10874 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:37.328578 10874 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:37.330438 10874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 640bb2ba46464896b30d48406ac663a2: Generated new TSK 0
I20260810 22:58:37.330636 10874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:37.363446 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:37.446579 10890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:37.446830 10890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:37.446895 10890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:37.448096 10890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:37.448164 10890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:37.448218 10890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:37.449743 10890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:37.450021 10890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:37.450250 10890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:37.455267 10899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.455312 10897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.455320 10903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.455597 10890 server_base.cc:1034] running on GCE node
I20260810 22:58:37.455829 10890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:37.456066 10890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:37.457237 10890 hybrid_clock.cc:648] HybridClock initialized: now 1786402717457210 us; error 48 us; skew 500 ppm
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:37.459728 10890 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:37.460222 10890 webserver.cc:460] Webserver started at http://127.6.47.129:39515/ using document root <none> and password file <none>
I20260810 22:58:37.460469 10890 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:37.460570 10890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:37.460759 10890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:37.461958 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "b635150bb24c4e9b939a0b8424905690"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "ed3d8063e5da8824ef1187e3495aa5f7"
server_key_iv: "30ef487213932fe9d0335cedb345a76b"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.462345 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "b635150bb24c4e9b939a0b8424905690"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "ed3d8063e5da8824ef1187e3495aa5f7"
server_key_iv: "30ef487213932fe9d0335cedb345a76b"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.463858 10890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:37.464653 10913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.464862 10890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:37.464977 10890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "b635150bb24c4e9b939a0b8424905690"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "ed3d8063e5da8824ef1187e3495aa5f7"
server_key_iv: "30ef487213932fe9d0335cedb345a76b"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.465049 10890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:37.485699 10890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:37.487494 10890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:37.487658 10890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:37.487896 10890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:37.488229 10890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:37.488283 10890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.488318 10890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:37.488344 10890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.494630 10890 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:44669
I20260810 22:58:37.494696 11118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:44669 every 8 connection(s)
I20260810 22:58:37.494997 10890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:37.499440 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10890
I20260810 22:58:37.499539 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:37.499732 6334 external_mini_cluster.cc:1428] Setting key c717aa49cff0a20ec53badc963708fdd
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:37.504941 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.495670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:43923 (local address 127.6.47.129:50035)
0810 22:58:37.495864 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:37.495904 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:37.496383 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:37.496683 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:37.496687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:37.497238 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:37.497424 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.497429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.498066 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.498066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.498889 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.498892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.499292 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.499293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.499448 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.500036 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:37.500044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:37.501565 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:37.503585 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.503593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.503594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.503822 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.504034 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.504035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.504035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.504128 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.504410 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:37.504415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:37.504573 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 22:58:37.504663 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 22:58:37.504787 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260810 22:58:37.504940 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.495732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50035 (local address 127.6.47.190:43923)
0810 22:58:37.495812 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:37.495814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:37.495999 (+ 185us) server_negotiation.cc:407] Connection header received
0810 22:58:37.496477 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:37.496478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:37.496518 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:37.496585 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:37.497507 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.497982 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.498970 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.499184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.499256 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.501664 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:37.501666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:37.501666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:37.501674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:37.503493 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.503897 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.503899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.503899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.503964 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.504191 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.504192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.504192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.504338 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:37.504409 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:37.504707 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 22:58:37.504802 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:58:37.505785 11119 heartbeater.cc:347] Connected to a master server at 127.6.47.190:43923
I20260810 22:58:37.505893 11119 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:37.506102 11119 heartbeater.cc:511] Master 127.6.47.190:43923 requested a full tablet report, sending...
I20260810 22:58:37.506654 10768 ts_manager.cc:194] Registered new tserver with Master: b635150bb24c4e9b939a0b8424905690 (127.6.47.129:44669)
I20260810 22:58:37.507259 10768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:50035
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:37.526736 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:37.609467 11127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:37.609673 11127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:37.609695 11127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:37.610880 11127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:37.610925 11127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:37.610980 11127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:37.612435 11127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:37.612708 11127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:37.612949 11127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:37.618074 11135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.618265 11127 server_base.cc:1034] running on GCE node
W20260810 22:58:37.618065 11133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.618065 11139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.618606 11127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:37.618817 11127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:37.619961 11127 hybrid_clock.cc:648] HybridClock initialized: now 1786402717619950 us; error 38 us; skew 500 ppm
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:37.622589 11127 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:37.623090 11127 webserver.cc:460] Webserver started at http://127.6.47.130:45889/ using document root <none> and password file <none>
I20260810 22:58:37.623342 11127 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:37.623411 11127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:37.623574 11127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:37.624634 11127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "6ca367b4dd8c4c9fab9941d1eda02aae"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "c195d31e4111a71f26d0f4fd40b44c40"
server_key_iv: "39547b6559c18bece7c0a0392a36706a"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.625007 11127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "6ca367b4dd8c4c9fab9941d1eda02aae"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "c195d31e4111a71f26d0f4fd40b44c40"
server_key_iv: "39547b6559c18bece7c0a0392a36706a"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.626323 11127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:37.627049 11152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.627246 11127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.627364 11127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "6ca367b4dd8c4c9fab9941d1eda02aae"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "c195d31e4111a71f26d0f4fd40b44c40"
server_key_iv: "39547b6559c18bece7c0a0392a36706a"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.627435 11127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:37.639515 11127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:37.641891 11127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:37.642055 11127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:37.642279 11127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:37.642627 11127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:37.642680 11127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.642716 11127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:37.642745 11127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.648762 11127 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:46735
I20260810 22:58:37.648833 11353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:46735 every 8 connection(s)
I20260810 22:58:37.649129 11127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:37.651737 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11127
I20260810 22:58:37.651834 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:37.652031 6334 external_mini_cluster.cc:1428] Setting key ebbff9346b3b8d350cfaded76a9e666a
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:37.659272 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.649873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:54867 (local address 127.6.47.190:43923)
0810 22:58:37.649950 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:37.649952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:37.650160 (+ 208us) server_negotiation.cc:407] Connection header received
0810 22:58:37.650666 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:37.650668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:37.650681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:37.650745 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:37.651705 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.652388 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.653510 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.653828 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.653909 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.656342 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:37.656344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:37.656344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:37.656351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:37.657785 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.658289 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.658290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.658290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.658341 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.658645 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.658646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.658646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.658742 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:37.658792 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:37.659145 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 22:58:37.659208 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:58:37.659596 11357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.649858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:43923 (local address 127.6.47.130:54867)
0810 22:58:37.650037 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:37.650070 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:37.650555 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:37.650821 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:37.650824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:37.651375 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:37.651554 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.651559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.652481 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.652483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.653411 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.653414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.653920 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.653921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.654024 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.654538 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:37.654546 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:37.656239 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:37.657872 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.657880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.657882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.658211 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.658413 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.658414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.658415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.658573 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.658810 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:37.658818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:37.658992 (+ 174us) client_negotiation.cc:769] Sending connection context
0810 22:58:37.659134 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 22:58:37.659319 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260810 22:58:37.660152 11354 heartbeater.cc:347] Connected to a master server at 127.6.47.190:43923
I20260810 22:58:37.660259 11354 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:37.660475 11354 heartbeater.cc:511] Master 127.6.47.190:43923 requested a full tablet report, sending...
I20260810 22:58:37.660970 10768 ts_manager.cc:194] Registered new tserver with Master: 6ca367b4dd8c4c9fab9941d1eda02aae (127.6.47.130:46735)
I20260810 22:58:37.661435 10768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:54867
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:37.679870 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:37.764919 11362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:37.765105 11362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:37.765127 11362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:37.766177 11362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:37.766211 11362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:37.766237 11362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:37.767599 11362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:43923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:37.767778 11362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:37.767943 11362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:37.772933 11382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.772933 11379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:37.772958 11373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:37.773162 11362 server_base.cc:1034] running on GCE node
I20260810 22:58:37.773408 11362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:37.773609 11362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:37.774771 11362 hybrid_clock.cc:648] HybridClock initialized: now 1786402717774744 us; error 26 us; skew 500 ppm
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:37.777251 11362 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:37.777683 11362 webserver.cc:460] Webserver started at http://127.6.47.131:46801/ using document root <none> and password file <none>
I20260810 22:58:37.777920 11362 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:37.777988 11362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:37.778142 11362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:37.779132 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "f84d1b69f0c841cf946cb68c48d83e75"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "2d8f654f8e67c7ef73fb6b47c6c19f9b"
server_key_iv: "5ea2ddba8d08b19f231d165c8d1fdc5e"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.779507 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "f84d1b69f0c841cf946cb68c48d83e75"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "2d8f654f8e67c7ef73fb6b47c6c19f9b"
server_key_iv: "5ea2ddba8d08b19f231d165c8d1fdc5e"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.780840 11362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:37.781574 11392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.781744 11362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:37.781852 11362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "f84d1b69f0c841cf946cb68c48d83e75"
format_stamp: "Formatted at 2026-08-10 22:58:37 on dist-test-slave-rwrg"
server_key: "2d8f654f8e67c7ef73fb6b47c6c19f9b"
server_key_iv: "5ea2ddba8d08b19f231d165c8d1fdc5e"
server_key_version: "encryptionkey@0"
I20260810 22:58:37.781927 11362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:37.794689 11362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:37.796483 11362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:37.796705 11362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:37.796965 11362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:37.797277 11362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:37.797310 11362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.797331 11362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:37.797343 11362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:37.804178 11362 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:46717
I20260810 22:58:37.804245 11583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:46717 every 8 connection(s)
I20260810 22:58:37.804641 11362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:37.805150 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11362
I20260810 22:58:37.805262 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:37.805449 6334 external_mini_cluster.cc:1428] Setting key 07a54f65a44dedc559d1416decebb5b1
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[10689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402717, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:37.814078 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.806025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:50051 (local address 127.6.47.190:43923)
0810 22:58:37.806067 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:37.806069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:37.806368 (+ 299us) server_negotiation.cc:407] Connection header received
0810 22:58:37.806858 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:37.806859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:37.806869 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:37.806915 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:37.807813 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.808279 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.809365 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.809545 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.809599 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.811714 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:37.811716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:37.811716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:37.811722 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:37.812941 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.813278 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.813278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.813279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.813346 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:37.813560 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:37.813560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:37.813561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:37.813665 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:37.813715 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:37.813955 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 22:58:37.814014 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:37.814284 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:37.806035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:43923 (local address 127.6.47.131:50051)
0810 22:58:37.806200 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:37.806236 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:37.806774 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:37.806968 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:37.806971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:37.807515 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:37.807723 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.807727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.808361 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.808362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.809261 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:37.809264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:37.809621 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:37.809622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:37.809726 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:37.810272 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:37.810280 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:37.811630 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:37.813011 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.813017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.813018 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.813215 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.813411 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:37.813412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:37.813412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:37.813506 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:37.813720 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:37.813724 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:37.813845 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 22:58:37.813924 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 22:58:37.814036 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260810 22:58:37.814786 11585 heartbeater.cc:347] Connected to a master server at 127.6.47.190:43923
I20260810 22:58:37.814908 11585 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:37.815083 11585 heartbeater.cc:511] Master 127.6.47.190:43923 requested a full tablet report, sending...
I20260810 22:58:37.815471 10768 ts_manager.cc:194] Registered new tserver with Master: f84d1b69f0c841cf946cb68c48d83e75 (127.6.47.131:46717)
I20260810 22:58:37.815828 10768 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:50051
I20260810 22:58:37.817216 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:58:37.830156 10848 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:37.829179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52572 (local address 127.6.47.190:43923)
0810 22:58:37.829250 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:37.829252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:37.829374 (+ 122us) server_negotiation.cc:407] Connection header received
0810 22:58:37.829471 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:37.829472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:37.829482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:37.829535 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:37.830063 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52572: BlockingRecv error: recv got EOF from 127.0.0.1:52572 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 22:58:37.831090 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10890
I20260810 22:58:37.847112 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11127
I20260810 22:58:37.861886 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11362
I20260810 22:58:37.874976 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10708
2026-08-10T22:58:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2185 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[11614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[11614](info): set up 2 sockets
Aug 10 22:58:37 dist-test-slave-rwrg krb5kdc[11614](info): commencing operation
krb5kdc: starting...
W20260810 22:58:39.162358 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:39.283740 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37217
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:37217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:39.366880 11636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:39.367126 11636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:39.367163 11636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:39.368285 11636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:39.368355 11636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:39.368381 11636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:39.368409 11636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:39.368439 11636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:39.370033 11636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:37217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:39.370325 11636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:39.370568 11636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:39.375653 11644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:39.375648 11646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:39.375746 11643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:39.375918 11636 server_base.cc:1034] running on GCE node
I20260810 22:58:39.376212 11636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:39.376544 11636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:39.377709 11636 hybrid_clock.cc:648] HybridClock initialized: now 1786402719377698 us; error 42 us; skew 500 ppm
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402719, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:39.383490 11636 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:39.383992 11636 webserver.cc:460] Webserver started at http://127.6.47.190:38065/ using document root <none> and password file <none>
I20260810 22:58:39.384262 11636 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:39.384359 11636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:39.384562 11636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:39.385735 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "3c2aee3efd0a41378f45ee05329e3d1f"
format_stamp: "Formatted at 2026-08-10 22:58:39 on dist-test-slave-rwrg"
server_key: "8dea3ff3d9b7886c4e8d4afd739cfee5"
server_key_iv: "6f535284c3f3c19bb73843d855f77b63"
server_key_version: "encryptionkey@0"
I20260810 22:58:39.386114 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "3c2aee3efd0a41378f45ee05329e3d1f"
format_stamp: "Formatted at 2026-08-10 22:58:39 on dist-test-slave-rwrg"
server_key: "8dea3ff3d9b7886c4e8d4afd739cfee5"
server_key_iv: "6f535284c3f3c19bb73843d855f77b63"
server_key_version: "encryptionkey@0"
I20260810 22:58:39.387508 11636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:39.388656 11657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:39.388870 11636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:39.388954 11636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "3c2aee3efd0a41378f45ee05329e3d1f"
format_stamp: "Formatted at 2026-08-10 22:58:39 on dist-test-slave-rwrg"
server_key: "8dea3ff3d9b7886c4e8d4afd739cfee5"
server_key_iv: "6f535284c3f3c19bb73843d855f77b63"
server_key_version: "encryptionkey@0"
I20260810 22:58:39.389010 11636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:39.407717 11636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:39.410177 11636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:39.410290 11636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:39.414286 11636 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:37217
I20260810 22:58:39.414335 11747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:37217 every 8 connection(s)
I20260810 22:58:39.414680 11636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:39.415581 11748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:39.417896 11748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f: Bootstrap starting.
I20260810 22:58:39.418531 11748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:39.418835 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11636
I20260810 22:58:39.418857 11748 log.cc:824] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:39.418910 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:39.419085 6334 external_mini_cluster.cc:1428] Setting key a7c015d9f39da24664a760d759b6d4cf
I20260810 22:58:39.419710 11748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f: No bootstrap required, opened a new log
I20260810 22:58:39.422359 11748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } }
I20260810 22:58:39.422521 11748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:39.422556 11748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2aee3efd0a41378f45ee05329e3d1f, State: Initialized, Role: FOLLOWER
I20260810 22:58:39.422675 11748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } }
I20260810 22:58:39.422745 11748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:39.422770 11748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:39.422811 11748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:39.423435 11748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } }
I20260810 22:58:39.423563 11748 leader_election.cc:302] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c2aee3efd0a41378f45ee05329e3d1f; no voters:
I20260810 22:58:39.423799 11748 leader_election.cc:288] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:39.423874 11754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:39.424101 11754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [term 1 LEADER]: Becoming Leader. State: Replica: 3c2aee3efd0a41378f45ee05329e3d1f, State: Running, Role: LEADER
I20260810 22:58:39.424105 11748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:39.424259 11754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } }
I20260810 22:58:39.424655 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } } }
I20260810 22:58:39.424769 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:39.424654 11758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c2aee3efd0a41378f45ee05329e3d1f. Latest consensus state: current_term: 1 leader_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c2aee3efd0a41378f45ee05329e3d1f" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37217 } } }
I20260810 22:58:39.424834 11758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:39.425086 11769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:39.425611 11769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:39.427784 11769 catalog_manager.cc:1347] Generated new cluster ID: 362f1d90342e4e26b456f24f9faa37cb
I20260810 22:58:39.427883 11769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:39.429862 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:39.419611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.6.47.190:37217)
0810 22:58:39.419786 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:39.419795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:39.419827 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:58:39.420269 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:39.420275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:39.420316 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:39.420422 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:39.421287 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:39.422146 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:39.423146 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:39.423519 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:39.423650 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:39.426616 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:39.426620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:39.426621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:39.426633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:39.428275 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:39.428713 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:39.428717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:39.428717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:39.428805 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:39.429018 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:39.429019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:39.429019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:39.429260 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:39.429449 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:39.429528 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 22:58:39.429667 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260810 22:58:39.430814 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:58:39.446235 11769 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:39.447009 11769 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:39.450883 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:39.446549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57838 (local address 127.6.47.190:37217)
0810 22:58:39.446607 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:39.446611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:39.446781 (+ 170us) server_negotiation.cc:407] Connection header received
0810 22:58:39.446848 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:39.446849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:39.447010 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:39.447081 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:39.447515 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:39.448209 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:39.449290 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:39.449587 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:39.449644 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:39.450373 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:39.450374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:39.450375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:39.450614 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:39.450631 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:39.450637 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:39.450700 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:39.450783 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 22:58:39.450808 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:58:39.454093 11769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c2aee3efd0a41378f45ee05329e3d1f: Generated new TSK 0
I20260810 22:58:39.454295 11769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:58:39.456678 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11636
2026-08-10T22:58:39Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1583 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11807](info): set up 2 sockets
Aug 10 22:58:39 dist-test-slave-rwrg krb5kdc[11807](info): commencing operation
krb5kdc: starting...
W20260810 22:58:40.739604 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:40 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:40.864959 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40761
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:40761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:40.946790 11829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:40.946981 11829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:40.947003 11829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:40.948153 11829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:40.948202 11829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:40.948216 11829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:40.948230 11829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:40.948282 11829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:40.949873 11829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:40761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:40.950125 11829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:40.950357 11829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 22:58:40.955368 11829 server_base.cc:1034] running on GCE node
W20260810 22:58:40.955259 11841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:40.955286 11835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:40.955260 11837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:40.955760 11829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:40.955993 11829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:40.957134 11829 hybrid_clock.cc:648] HybridClock initialized: now 1786402720957111 us; error 36 us; skew 500 ppm
Aug 10 22:58:40 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402720, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:40.959664 11829 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:40.960103 11829 webserver.cc:460] Webserver started at http://127.6.47.190:43055/ using document root <none> and password file <none>
I20260810 22:58:40.960281 11829 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:40.960319 11829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:40.960409 11829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:40.961514 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "69bbdbfd2b1945cc9c3f93ae80e85391"
format_stamp: "Formatted at 2026-08-10 22:58:40 on dist-test-slave-rwrg"
server_key: "7f3622671f4a0f7ae96eacb21ed01d5e"
server_key_iv: "ef7f68608bb2f74fa419b4de65b6ae9f"
server_key_version: "encryptionkey@0"
I20260810 22:58:40.961828 11829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "69bbdbfd2b1945cc9c3f93ae80e85391"
format_stamp: "Formatted at 2026-08-10 22:58:40 on dist-test-slave-rwrg"
server_key: "7f3622671f4a0f7ae96eacb21ed01d5e"
server_key_iv: "ef7f68608bb2f74fa419b4de65b6ae9f"
server_key_version: "encryptionkey@0"
I20260810 22:58:40.963085 11829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:40.963802 11856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:40.964001 11829 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:40.964133 11829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "69bbdbfd2b1945cc9c3f93ae80e85391"
format_stamp: "Formatted at 2026-08-10 22:58:40 on dist-test-slave-rwrg"
server_key: "7f3622671f4a0f7ae96eacb21ed01d5e"
server_key_iv: "ef7f68608bb2f74fa419b4de65b6ae9f"
server_key_version: "encryptionkey@0"
I20260810 22:58:40.964211 11829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:40.978874 11829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:40.980844 11829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:40.981016 11829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:40.985119 11829 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:40761
I20260810 22:58:40.985155 11956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:40761 every 8 connection(s)
I20260810 22:58:40.985494 11829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:40.986526 11957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:40.988814 11957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391: Bootstrap starting.
I20260810 22:58:40.989399 11957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:40.989697 11957 log.cc:824] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:40.989960 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11829
I20260810 22:58:40.990073 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:40.990252 6334 external_mini_cluster.cc:1428] Setting key 551c084d35602550c344869834fa3774
I20260810 22:58:40.990525 11957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391: No bootstrap required, opened a new log
I20260810 22:58:40.992854 11957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } }
I20260810 22:58:40.992981 11957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:40.993005 11957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69bbdbfd2b1945cc9c3f93ae80e85391, State: Initialized, Role: FOLLOWER
I20260810 22:58:40.993132 11957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } }
I20260810 22:58:40.993237 11957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:40.993261 11957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:40.993355 11957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:40.993898 11957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } }
I20260810 22:58:40.993988 11957 leader_election.cc:302] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69bbdbfd2b1945cc9c3f93ae80e85391; no voters:
I20260810 22:58:40.994192 11957 leader_election.cc:288] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:40.994238 11967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:40.994411 11967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [term 1 LEADER]: Becoming Leader. State: Replica: 69bbdbfd2b1945cc9c3f93ae80e85391, State: Running, Role: LEADER
I20260810 22:58:40.994446 11957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:40.994553 11967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } }
I20260810 22:58:40.995059 11967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69bbdbfd2b1945cc9c3f93ae80e85391. Latest consensus state: current_term: 1 leader_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } } }
I20260810 22:58:40.995066 11971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69bbdbfd2b1945cc9c3f93ae80e85391" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40761 } } }
I20260810 22:58:40.995221 11967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:40.995231 11971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:40.995460 11982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:40.996025 11982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 22:58:40 dist-test-slave-rwrg krb5kdc[11807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:41.001714 11982 catalog_manager.cc:1347] Generated new cluster ID: 59113a89991f48628de5c5e65fcb187d
I20260810 22:58:41.001787 11982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:41.001950 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:40.990746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.6.47.190:40761)
0810 22:58:40.990926 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:40.990934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:40.990959 (+ 25us) server_negotiation.cc:407] Connection header received
0810 22:58:40.991789 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:40.991796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:40.991843 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:40.991954 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:40.992788 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:40.994116 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:40.995323 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:40.995956 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:40.996068 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:40.998602 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:40.998618 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:40.998618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:40.998628 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:41.000225 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.000631 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.000634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.000634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.000723 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.000949 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.000950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.000951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.001287 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:41.001476 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:41.001556 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 22:58:41.001708 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260810 22:58:41.019111 11982 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:41.019868 11982 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:41.029503 11982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69bbdbfd2b1945cc9c3f93ae80e85391: Generated new TSK 0
I20260810 22:58:41.029738 11982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:41.063339 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:41.160991 12004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:41.161213 12004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:41.161238 12004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:41.162312 12004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:41.162345 12004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:41.162370 12004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:41.163763 12004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:41.163956 12004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:41.164113 12004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:41.169215 12015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.169229 12016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.169315 12018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:41.169580 12004 server_base.cc:1034] running on GCE node
I20260810 22:58:41.169799 12004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:41.170037 12004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:41.171202 12004 hybrid_clock.cc:648] HybridClock initialized: now 1786402721171172 us; error 61 us; skew 500 ppm
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:41.173771 12004 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:41.174199 12004 webserver.cc:460] Webserver started at http://127.6.47.129:38205/ using document root <none> and password file <none>
I20260810 22:58:41.174419 12004 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:41.174482 12004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:41.174688 12004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:41.175711 12004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "ab0920fac9a04152b613d129258792e6"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "c77179f36fa47ea4f50421202348fba2"
server_key_iv: "5d456f40d50b735943091c8e2ca8e023"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.176141 12004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "ab0920fac9a04152b613d129258792e6"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "c77179f36fa47ea4f50421202348fba2"
server_key_iv: "5d456f40d50b735943091c8e2ca8e023"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.177526 12004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:41.178249 12031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.178411 12004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:41.178514 12004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "ab0920fac9a04152b613d129258792e6"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "c77179f36fa47ea4f50421202348fba2"
server_key_iv: "5d456f40d50b735943091c8e2ca8e023"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.178584 12004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:41.208086 12004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:41.210800 12004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:41.210990 12004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:41.211349 12004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:41.211736 12004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:41.211779 12004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.211855 12004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:41.211885 12004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.218328 12004 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:34151
I20260810 22:58:41.218396 12240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:34151 every 8 connection(s)
I20260810 22:58:41.218729 12004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:41.219520 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12004
I20260810 22:58:41.219620 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:41.219843 6334 external_mini_cluster.cc:1428] Setting key ed5b53d9458e548edf2e0b0a0962d188
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:41.229774 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.219424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40761 (local address 127.6.47.129:33089)
0810 22:58:41.219607 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:41.219658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:41.220377 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:41.220653 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:41.220656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:41.221227 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:41.221431 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.221436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.222428 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.222430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.223823 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.223827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.224322 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.224323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.224426 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.224975 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:41.224984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:41.226420 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:41.228335 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.228342 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.228344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.228632 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.228856 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.228857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.228857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.228955 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.229303 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:41.229308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:41.229418 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 22:58:41.229502 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:58:41.229618 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260810 22:58:41.229794 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.219430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:33089 (local address 127.6.47.190:40761)
0810 22:58:41.219477 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:41.219479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:41.219750 (+ 271us) server_negotiation.cc:407] Connection header received
0810 22:58:41.220487 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:41.220489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:41.220521 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:41.220576 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:41.221534 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.222316 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.223913 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.224238 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.224310 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.226517 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:41.226519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:41.226519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:41.226528 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:41.228248 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.228708 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.228710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.228710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.228779 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.229030 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.229031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.229031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.229213 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:41.229308 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:41.229557 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 22:58:41.229658 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
I20260810 22:58:41.230407 12243 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40761
I20260810 22:58:41.230620 12243 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:41.230834 12243 heartbeater.cc:511] Master 127.6.47.190:40761 requested a full tablet report, sending...
I20260810 22:58:41.231426 11890 ts_manager.cc:194] Registered new tserver with Master: ab0920fac9a04152b613d129258792e6 (127.6.47.129:34151)
I20260810 22:58:41.232066 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:33089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:41.247560 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:41.350342 12250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:41.350580 12250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:41.350620 12250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:41.351871 12250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:41.351938 12250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:41.351981 12250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:41.353541 12250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:41.353820 12250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:41.354070 12250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:41.359207 12255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.359231 12258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.359386 12263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:41.359443 12250 server_base.cc:1034] running on GCE node
I20260810 22:58:41.359675 12250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:41.359869 12250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:41.361008 12250 hybrid_clock.cc:648] HybridClock initialized: now 1786402721360997 us; error 28 us; skew 500 ppm
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:41.363547 12250 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:41.363993 12250 webserver.cc:460] Webserver started at http://127.6.47.130:40423/ using document root <none> and password file <none>
I20260810 22:58:41.364256 12250 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:41.364327 12250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:41.364530 12250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:41.365621 12250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "21adda21688346aea91ae7b9ef1ce339"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "7b8522373f72799a3459b1a9117dfa1b"
server_key_iv: "db156fc3f35eb7f4171d88a1e8b164e7"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.366017 12250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "21adda21688346aea91ae7b9ef1ce339"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "7b8522373f72799a3459b1a9117dfa1b"
server_key_iv: "db156fc3f35eb7f4171d88a1e8b164e7"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.367470 12250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:41.368245 12281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.368427 12250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:41.368551 12250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "21adda21688346aea91ae7b9ef1ce339"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "7b8522373f72799a3459b1a9117dfa1b"
server_key_iv: "db156fc3f35eb7f4171d88a1e8b164e7"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.368642 12250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:41.385027 12250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:41.387640 12250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:41.387840 12250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:41.388128 12250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:41.388557 12250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:41.388648 12250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.388724 12250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:41.388780 12250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.396080 12250 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:41091
I20260810 22:58:41.396166 12467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:41091 every 8 connection(s)
I20260810 22:58:41.396477 12250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:41.403327 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12250
I20260810 22:58:41.403435 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:41.403630 6334 external_mini_cluster.cc:1428] Setting key 51af081d155853b01e739b833b57d031
I20260810 22:58:41.407328 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.397242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40761 (local address 127.6.47.130:58573)
0810 22:58:41.397431 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:41.397468 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:41.397991 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:41.398252 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:41.398256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:41.398832 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:41.399031 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.399036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.399747 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.399749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.400636 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.400639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.401047 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.401048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.401182 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.401749 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:41.401758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:41.403633 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:41.405713 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.405721 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.405723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.406065 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.406339 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.406340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.406341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.406465 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.406846 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:41.406851 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:41.406965 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 22:58:41.407050 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 22:58:41.407169 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":97,"threads_started":1}
I20260810 22:58:41.407362 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.397278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:58573 (local address 127.6.47.190:40761)
0810 22:58:41.397322 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:41.397324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:41.397552 (+ 228us) server_negotiation.cc:407] Connection header received
0810 22:58:41.398108 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:41.398109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:41.398123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:41.398181 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:41.399133 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.399644 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.400731 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.400945 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.401034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.403793 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:41.403795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:41.403795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:41.403816 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:41.405611 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.406158 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.406159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.406160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.406244 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.406583 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.406584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.406584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.406763 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:41.406857 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:41.407110 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 22:58:41.407221 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:58:41.407863 12470 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40761
I20260810 22:58:41.408018 12470 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:41.408236 12470 heartbeater.cc:511] Master 127.6.47.190:40761 requested a full tablet report, sending...
I20260810 22:58:41.408802 11890 ts_manager.cc:194] Registered new tserver with Master: 21adda21688346aea91ae7b9ef1ce339 (127.6.47.130:41091)
I20260810 22:58:41.409344 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:58573
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:41.431288 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:41.521134 12485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:41.521344 12485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:41.521368 12485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:41.522679 12485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:41.522738 12485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:41.522796 12485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:41.524345 12485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:41.524616 12485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:41.524896 12485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:41.530099 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.530076 12500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:41.530067 12491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:41.530400 12485 server_base.cc:1034] running on GCE node
I20260810 22:58:41.530603 12485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:41.530804 12485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:41.531939 12485 hybrid_clock.cc:648] HybridClock initialized: now 1786402721531928 us; error 28 us; skew 500 ppm
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:41.534677 12485 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:41.535171 12485 webserver.cc:460] Webserver started at http://127.6.47.131:35899/ using document root <none> and password file <none>
I20260810 22:58:41.535439 12485 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:41.535537 12485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:41.535652 12485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:41.536755 12485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "08f33a4cb9354c94ab26d325cc9c1f6b"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "035497c373f1cf5f37077bc217493a27"
server_key_iv: "0345e8f014bb5f0761e391d4c51d9a00"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.537165 12485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "08f33a4cb9354c94ab26d325cc9c1f6b"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "035497c373f1cf5f37077bc217493a27"
server_key_iv: "0345e8f014bb5f0761e391d4c51d9a00"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.538544 12485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:41.539261 12512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.539441 12485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:41.539603 12485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "08f33a4cb9354c94ab26d325cc9c1f6b"
format_stamp: "Formatted at 2026-08-10 22:58:41 on dist-test-slave-rwrg"
server_key: "035497c373f1cf5f37077bc217493a27"
server_key_iv: "0345e8f014bb5f0761e391d4c51d9a00"
server_key_version: "encryptionkey@0"
I20260810 22:58:41.539690 12485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:41.557016 12485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:41.558836 12485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:41.558997 12485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:41.559226 12485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:41.559537 12485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:41.559593 12485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.559636 12485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:41.559664 12485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:41.565662 12485 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:40915
I20260810 22:58:41.565711 12719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:40915 every 8 connection(s)
I20260810 22:58:41.566104 12485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:41.566474 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12485
I20260810 22:58:41.566562 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:41.566811 6334 external_mini_cluster.cc:1428] Setting key 297ebde959dbe5751d2d51e83d63100d
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:41.575417 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.566865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:55867 (local address 127.6.47.190:40761)
0810 22:58:41.566904 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:41.566906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:41.567157 (+ 251us) server_negotiation.cc:407] Connection header received
0810 22:58:41.567784 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:41.567785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:41.567797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:41.567857 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:41.568890 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.569402 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.570337 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.570523 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.570570 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.572854 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:41.572855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:41.572855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:41.572862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:41.574308 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.574657 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.574658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.574658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.574709 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.574930 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.574931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.574931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.575028 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:41.575076 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:41.575297 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 22:58:41.575356 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:41.575570 12722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.566852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40761 (local address 127.6.47.131:55867)
0810 22:58:41.567023 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:41.567069 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:41.567638 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:41.567959 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:41.567963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:41.568612 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:41.568815 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.568820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.569482 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.569483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.570265 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:41.570267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.570593 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.570594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:41.570705 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.571223 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:41.571232 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:41.572756 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:41.574382 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.574389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.574390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.574592 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.574770 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:41.574772 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:41.574773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:41.574873 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:41.575081 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:41.575085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:41.575190 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 22:58:41.575269 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 22:58:41.575381 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260810 22:58:41.576047 12720 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40761
I20260810 22:58:41.576162 12720 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:41.576349 12720 heartbeater.cc:511] Master 127.6.47.190:40761 requested a full tablet report, sending...
I20260810 22:58:41.576772 11890 ts_manager.cc:194] Registered new tserver with Master: 08f33a4cb9354c94ab26d325cc9c1f6b (127.6.47.131:40915)
I20260810 22:58:41.577157 11890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:55867
I20260810 22:58:41.578619 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[11807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402721, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:41.603353 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:41.595908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.6.47.190:40761)
0810 22:58:41.595975 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:41.596004 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:41.596102 (+ 98us) server_negotiation.cc:407] Connection header received
0810 22:58:41.596204 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:41.596205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:41.596216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:41.596290 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:41.596945 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.597419 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.598162 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:41.598368 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:41.598445 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:41.600681 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:41.600682 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:41.600683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:41.600691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:41.602104 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.602499 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.602500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.602500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.602561 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:41.602779 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:41.602780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:41.602780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:41.602943 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:41.603050 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:41.603155 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 22:58:41.603242 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 22:58:41.603698 11890 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60256
I20260810 22:58:41.604115 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12004
I20260810 22:58:41.614529 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12250
I20260810 22:58:41.625093 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12485
I20260810 22:58:41.637408 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11829
2026-08-10T22:58:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2179 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[12739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[12739](info): set up 2 sockets
Aug 10 22:58:41 dist-test-slave-rwrg krb5kdc[12739](info): commencing operation
krb5kdc: starting...
W20260810 22:58:42.927886 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:43.048621 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39263
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:39263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:43.130625 12767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:43.130878 12767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:43.130916 12767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:43.132062 12767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:43.132133 12767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:43.132158 12767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:43.132179 12767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:43.132206 12767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:43.133806 12767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:39263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:43.134096 12767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:43.134326 12767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:43.139160 12781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.139161 12775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.139351 12767 server_base.cc:1034] running on GCE node
W20260810 22:58:43.139204 12776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.139657 12767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:43.139911 12767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:43.141072 12767 hybrid_clock.cc:648] HybridClock initialized: now 1786402723141058 us; error 46 us; skew 500 ppm
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:43.143515 12767 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:43.143965 12767 webserver.cc:460] Webserver started at http://127.6.47.190:37341/ using document root <none> and password file <none>
I20260810 22:58:43.144201 12767 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:43.144271 12767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:43.144428 12767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:43.145520 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "5024955d265fd2309b557ec3a6514026"
server_key_iv: "8ca40028f2ecc16bd970b02e2e7628e4"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.145898 12767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "5024955d265fd2309b557ec3a6514026"
server_key_iv: "8ca40028f2ecc16bd970b02e2e7628e4"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.147225 12767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:43.147948 12795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.148128 12767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:43.148237 12767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "5024955d265fd2309b557ec3a6514026"
server_key_iv: "8ca40028f2ecc16bd970b02e2e7628e4"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.148322 12767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:43.169648 12767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:43.171551 12767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:43.171736 12767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:43.176030 12767 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:39263
I20260810 22:58:43.176066 12893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:39263 every 8 connection(s)
I20260810 22:58:43.176679 12767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:43.177397 12894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:43.179577 12894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c: Bootstrap starting.
I20260810 22:58:43.180125 12894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:43.180378 12894 log.cc:824] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:43.181088 12894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c: No bootstrap required, opened a new log
I20260810 22:58:43.182623 12894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } }
I20260810 22:58:43.182760 12894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:43.182781 12894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94db50a3e7ae4edbbabdf30d48cc8a9c, State: Initialized, Role: FOLLOWER
I20260810 22:58:43.182912 12894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } }
I20260810 22:58:43.182993 12894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:43.183017 12894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:43.183045 12894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:43.183465 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12767
I20260810 22:58:43.183578 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:43.183777 6334 external_mini_cluster.cc:1428] Setting key 7a0ebf770c75f81ab17f54e98c7b6a0c
I20260810 22:58:43.184007 12894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } }
I20260810 22:58:43.184129 12894 leader_election.cc:302] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94db50a3e7ae4edbbabdf30d48cc8a9c; no voters:
I20260810 22:58:43.184348 12894 leader_election.cc:288] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:43.184427 12903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:43.184630 12903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [term 1 LEADER]: Becoming Leader. State: Replica: 94db50a3e7ae4edbbabdf30d48cc8a9c, State: Running, Role: LEADER
I20260810 22:58:43.184719 12894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:43.184762 12903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } }
I20260810 22:58:43.185174 12909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94db50a3e7ae4edbbabdf30d48cc8a9c. Latest consensus state: current_term: 1 leader_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } } }
I20260810 22:58:43.185190 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94db50a3e7ae4edbbabdf30d48cc8a9c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39263 } } }
I20260810 22:58:43.185388 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:43.185600 12909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:43.185938 12921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:43.186519 12921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:43.188344 12921 catalog_manager.cc:1347] Generated new cluster ID: 184ee59c936b4f3a9c19f07ee2e3cd3b
I20260810 22:58:43.188408 12921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:43.194666 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.184261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.6.47.190:39263)
0810 22:58:43.184449 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.184460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.184649 (+ 189us) server_negotiation.cc:407] Connection header received
0810 22:58:43.184780 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.184789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.184844 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.184993 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.185947 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.187153 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.188328 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.188902 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.188964 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.191444 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.191448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.191449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.191467 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.193163 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.193551 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.193554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.193555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.193669 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.193892 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.193893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.193894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.194125 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.194304 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.194371 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.194509 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260810 22:58:43.198493 12921 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:43.199150 12921 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:43.209229 12921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94db50a3e7ae4edbbabdf30d48cc8a9c: Generated new TSK 0
I20260810 22:58:43.209438 12921 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:43.233477 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:43.317090 12945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:43.317297 12945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:43.317320 12945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:43.318602 12945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:43.318650 12945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:43.318675 12945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:43.320230 12945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:43.320475 12945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:43.320775 12945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:43.325795 12962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.325824 12961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.325984 12945 server_base.cc:1034] running on GCE node
W20260810 22:58:43.325899 12968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.326290 12945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:43.326581 12945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:43.327757 12945 hybrid_clock.cc:648] HybridClock initialized: now 1786402723327738 us; error 43 us; skew 500 ppm
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:43.330538 12945 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:43.330924 12945 webserver.cc:460] Webserver started at http://127.6.47.129:43757/ using document root <none> and password file <none>
I20260810 22:58:43.331138 12945 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:43.331180 12945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:43.331275 12945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:43.332278 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "44e13a1af08043cebb7e893aa0569174"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "e8c8c38e6b64b9c643c5ae101860b15a"
server_key_iv: "a1336e6cfd1630e6a5f5360f6bbb996b"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.332670 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "44e13a1af08043cebb7e893aa0569174"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "e8c8c38e6b64b9c643c5ae101860b15a"
server_key_iv: "a1336e6cfd1630e6a5f5360f6bbb996b"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.333964 12945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:43.334682 12983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.334892 12945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:43.334967 12945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "44e13a1af08043cebb7e893aa0569174"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "e8c8c38e6b64b9c643c5ae101860b15a"
server_key_iv: "a1336e6cfd1630e6a5f5360f6bbb996b"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.335021 12945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:43.358953 12945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:43.361117 12945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:43.361240 12945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:43.361439 12945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:43.361721 12945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:43.361755 12945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.361778 12945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:43.361790 12945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.367602 12945 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40625
I20260810 22:58:43.367703 13187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40625 every 8 connection(s)
I20260810 22:58:43.368010 12945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:43.368835 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12945
I20260810 22:58:43.368943 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:43.369144 6334 external_mini_cluster.cc:1428] Setting key c2e2e9a4414e93ec69ef843a324a9b70
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
I20260810 22:58:43.379110 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.368847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45603 (local address 127.6.47.190:39263)
0810 22:58:43.368905 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.368906 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.369139 (+ 233us) server_negotiation.cc:407] Connection header received
0810 22:58:43.370371 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.370372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.370388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.370453 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.371074 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.371863 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.372974 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.373210 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.373289 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.375934 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.375936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.375937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.375949 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.377528 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.378035 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.378036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.378036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.378102 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.378370 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.378371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.378371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.378532 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.378623 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.378917 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.379012 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 22:58:43.379371 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.368825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39263 (local address 127.6.47.129:45603)
0810 22:58:43.369020 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:43.369058 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:43.370256 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:43.370530 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:43.370533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:43.370790 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:43.370977 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.370981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.371969 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.371972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.372870 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.372873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.373301 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.373302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.373431 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.374031 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:43.374040 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:43.375813 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:43.377676 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.377685 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.377687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.377933 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.378184 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.378185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.378185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.378301 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.378617 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:43.378625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:43.378774 (+ 149us) client_negotiation.cc:769] Sending connection context
0810 22:58:43.378881 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 22:58:43.379045 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260810 22:58:43.379957 13188 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39263
I20260810 22:58:43.380065 13188 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:43.380283 13188 heartbeater.cc:511] Master 127.6.47.190:39263 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
I20260810 22:58:43.380924 12835 ts_manager.cc:194] Registered new tserver with Master: 44e13a1af08043cebb7e893aa0569174 (127.6.47.129:40625)
I20260810 22:58:43.381572 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:45603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:43.395874 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:43.477541 13196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:43.477753 13196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:43.477777 13196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:43.478901 13196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:43.478942 13196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:43.478997 13196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:43.480481 13196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:43.480749 13196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:43.481001 13196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:43.486069 13204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.486032 13207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.486053 13205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.486399 13196 server_base.cc:1034] running on GCE node
I20260810 22:58:43.486599 13196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:43.486812 13196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:43.487950 13196 hybrid_clock.cc:648] HybridClock initialized: now 1786402723487937 us; error 28 us; skew 500 ppm
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:43.490621 13196 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:43.491093 13196 webserver.cc:460] Webserver started at http://127.6.47.130:40321/ using document root <none> and password file <none>
I20260810 22:58:43.491348 13196 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:43.491392 13196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:43.491559 13196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:43.492686 13196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "80d9409f6a4b409fa64a6a39ddfd3ab5"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "b9ab9a524d3b1a32bc948193d681e537"
server_key_iv: "59f093d424b87b203493b70ed971eaa0"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.493052 13196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "80d9409f6a4b409fa64a6a39ddfd3ab5"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "b9ab9a524d3b1a32bc948193d681e537"
server_key_iv: "59f093d424b87b203493b70ed971eaa0"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.494379 13196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:43.495105 13216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.495277 13196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:43.495391 13196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "80d9409f6a4b409fa64a6a39ddfd3ab5"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "b9ab9a524d3b1a32bc948193d681e537"
server_key_iv: "59f093d424b87b203493b70ed971eaa0"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.495467 13196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:43.504025 13196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:43.505985 13196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:43.506166 13196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:43.506407 13196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:43.506734 13196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:43.506793 13196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.506829 13196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:43.506856 13196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.513244 13196 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:43977
I20260810 22:58:43.513302 13421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:43977 every 8 connection(s)
I20260810 22:58:43.513620 13196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:43.520722 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13196
I20260810 22:58:43.520853 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:43.521080 6334 external_mini_cluster.cc:1428] Setting key 9381b0786711301896beabb9fcabcf1d
I20260810 22:58:43.523684 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.514379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:36821 (local address 127.6.47.190:39263)
0810 22:58:43.514467 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.514470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.514652 (+ 182us) server_negotiation.cc:407] Connection header received
0810 22:58:43.515621 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.515622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.515635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.515707 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.516352 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.516934 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.517905 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.518118 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.518189 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.520582 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.520583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.520584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.520593 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.522232 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.522681 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.522683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.522684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.522745 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.523022 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.523023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.523024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.523142 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.523218 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.523497 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.523587 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:43.523869 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.514336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39263 (local address 127.6.47.130:36821)
0810 22:58:43.514520 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:43.514565 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:43.515502 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:43.515784 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:43.515792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:43.516050 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:43.516250 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.516254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.517032 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.517033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.517811 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.517813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.518241 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.518242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.518356 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.518909 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:43.518918 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:43.520443 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:43.522343 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.522351 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.522353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.522584 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.522820 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.522820 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.522821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.522946 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.523209 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:43.523217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:43.523358 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 22:58:43.523464 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:58:43.523622 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260810 22:58:43.524333 13422 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39263
I20260810 22:58:43.524446 13422 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:43.524682 13422 heartbeater.cc:511] Master 127.6.47.190:39263 requested a full tablet report, sending...
I20260810 22:58:43.525101 12835 ts_manager.cc:194] Registered new tserver with Master: 80d9409f6a4b409fa64a6a39ddfd3ab5 (127.6.47.130:43977)
I20260810 22:58:43.525453 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:36821
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:43.547714 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:43.632097 13429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:43.632303 13429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:43.632326 13429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:43.633620 13429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:43.633673 13429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:43.633728 13429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:43.635237 13429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:43.635494 13429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:43.635725 13429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:43.640798 13445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.640870 13443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:43.640800 13440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:43.641110 13429 server_base.cc:1034] running on GCE node
I20260810 22:58:43.641390 13429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:43.641631 13429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:43.642822 13429 hybrid_clock.cc:648] HybridClock initialized: now 1786402723642814 us; error 41 us; skew 500 ppm
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:43.645232 13429 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:43.645710 13429 webserver.cc:460] Webserver started at http://127.6.47.131:34485/ using document root <none> and password file <none>
I20260810 22:58:43.645955 13429 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:43.646024 13429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:43.646183 13429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:43.647215 13429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "e5992b58b5984863985777051a53e7b4"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "71603381d40e340be94eda6f4bc4a82d"
server_key_iv: "b85ff109365952c2ccbc0a69e620b7e9"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.647585 13429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "e5992b58b5984863985777051a53e7b4"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "71603381d40e340be94eda6f4bc4a82d"
server_key_iv: "b85ff109365952c2ccbc0a69e620b7e9"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.649107 13429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 22:58:43.649870 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.650043 13429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:43.650153 13429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "e5992b58b5984863985777051a53e7b4"
format_stamp: "Formatted at 2026-08-10 22:58:43 on dist-test-slave-rwrg"
server_key: "71603381d40e340be94eda6f4bc4a82d"
server_key_iv: "b85ff109365952c2ccbc0a69e620b7e9"
server_key_version: "encryptionkey@0"
I20260810 22:58:43.650233 13429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:43.660738 13429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:43.662583 13429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:43.662770 13429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:43.663010 13429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:43.663401 13429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:43.663437 13429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.663460 13429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:43.663472 13429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:43.669698 13429 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:35439
I20260810 22:58:43.669773 13667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:35439 every 8 connection(s)
I20260810 22:58:43.670080 13429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:43.673343 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13429
I20260810 22:58:43.673432 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:43.673632 6334 external_mini_cluster.cc:1428] Setting key 5b4a19abfe241e21c364f04561ee8207
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:43.679785 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.670901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:34789 (local address 127.6.47.190:39263)
0810 22:58:43.670951 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.670953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.671157 (+ 204us) server_negotiation.cc:407] Connection header received
0810 22:58:43.672120 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.672121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.672133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.672189 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.672908 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.673387 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.674608 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.674835 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.674934 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.677095 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.677096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.677097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.677104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.678479 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.678947 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.678948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.678949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.678999 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.679248 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.679249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.679249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.679348 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.679400 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.679661 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.679721 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:43.679987 13674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.670848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39263 (local address 127.6.47.131:34789)
0810 22:58:43.671020 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:43.671064 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:43.671989 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:43.672256 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:43.672259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:43.672582 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:43.672824 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.672829 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.673530 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.673531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.674515 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:43.674518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.674944 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.674945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:43.675064 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.675516 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:43.675524 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:43.676999 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:43.678561 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.678568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.678570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.678872 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.679064 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:43.679065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:43.679065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:43.679172 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:43.679407 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:43.679411 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:43.679524 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 22:58:43.679607 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:58:43.679747 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260810 22:58:43.680513 13668 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39263
I20260810 22:58:43.680615 13668 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:43.680768 13668 heartbeater.cc:511] Master 127.6.47.190:39263 requested a full tablet report, sending...
I20260810 22:58:43.681193 12835 ts_manager.cc:194] Registered new tserver with Master: e5992b58b5984863985777051a53e7b4 (127.6.47.131:35439)
I20260810 22:58:43.681535 12835 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:34789
I20260810 22:58:43.685431 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:43.712392 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.702255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.6.47.129:40625)
0810 22:58:43.702433 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.702438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.702449 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:43.702477 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.702479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.702493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.702563 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.703079 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.703802 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.704597 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.705097 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.705189 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.708380 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.708392 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.708392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.708399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.711129 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.711509 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.711512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.711513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.711582 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.711800 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.711801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.711802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.711954 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.712123 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.712176 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.712260 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[12739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:43.720044 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:43.713792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.6.47.190:39263)
0810 22:58:43.713839 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:43.713840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:43.713898 (+ 58us) server_negotiation.cc:407] Connection header received
0810 22:58:43.713952 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:43.713953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:43.713962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:43.714006 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:43.714430 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.714889 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.715491 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:43.715668 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:43.715715 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:43.717809 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:43.717810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:43.717810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:43.717816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:43.718996 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.719345 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.719346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.719346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.719396 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:43.719560 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:43.719561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:43.719561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:43.719704 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:43.719776 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:43.719883 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 22:58:43.719944 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260810 22:58:43.720408 12835 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40306
I20260810 22:58:43.720803 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12945
I20260810 22:58:43.740868 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13196
I20260810 22:58:43.770964 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13429
I20260810 22:58:43.785408 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12767
2026-08-10T22:58:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2147 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[13693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[13693](info): set up 2 sockets
Aug 10 22:58:43 dist-test-slave-rwrg krb5kdc[13693](info): commencing operation
krb5kdc: starting...
W20260810 22:58:45.364315 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.552s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:45.492731 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42283
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:45.573530 13711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:45.573843 13711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:45.573885 13711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:45.575002 13711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:45.575076 13711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:45.575100 13711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:45.575124 13711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:45.575150 13711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:45.576750 13711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:45.577029 13711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:45.577257 13711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:45.582003 13721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:45.582044 13719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:45.582111 13711 server_base.cc:1034] running on GCE node
W20260810 22:58:45.582031 13718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:45.582494 13711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:45.582842 13711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:45.583988 13711 hybrid_clock.cc:648] HybridClock initialized: now 1786402725583970 us; error 39 us; skew 500 ppm
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:45.589524 13711 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:45.589912 13711 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:45.589990 13711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:45.590159 13711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:45.591239 13711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "aa0ca0fd423a412ba7a2b155662ab12a"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "290e7bb2ebc9a12b392182ef8181fa72"
server_key_iv: "aac4d52073bf4afaf425aaae4d8da536"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.591621 13711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "aa0ca0fd423a412ba7a2b155662ab12a"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "290e7bb2ebc9a12b392182ef8181fa72"
server_key_iv: "aac4d52073bf4afaf425aaae4d8da536"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.593029 13711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:45.593866 13728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.594070 13711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:45.594192 13711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "aa0ca0fd423a412ba7a2b155662ab12a"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "290e7bb2ebc9a12b392182ef8181fa72"
server_key_iv: "aac4d52073bf4afaf425aaae4d8da536"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.594269 13711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:45.603932 13711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:45.605749 13711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:45.605909 13711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:45.610052 13711 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42283
I20260810 22:58:45.610077 13829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42283 every 8 connection(s)
I20260810 22:58:45.610364 13711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:45.611358 13831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:45.613665 13831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: Bootstrap starting.
I20260810 22:58:45.614221 13831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:45.614464 13831 log.cc:824] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:45.615125 13831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: No bootstrap required, opened a new log
I20260810 22:58:45.616686 13831 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } }
I20260810 22:58:45.616820 13831 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:45.616842 13831 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0ca0fd423a412ba7a2b155662ab12a, State: Initialized, Role: FOLLOWER
I20260810 22:58:45.616976 13831 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } }
I20260810 22:58:45.617055 13831 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:45.617079 13831 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:45.617113 13831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:45.617919 13831 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } }
I20260810 22:58:45.617987 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13711
I20260810 22:58:45.618040 13831 leader_election.cc:302] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa0ca0fd423a412ba7a2b155662ab12a; no voters:
I20260810 22:58:45.618088 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:45.618233 13831 leader_election.cc:288] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:45.618279 6334 external_mini_cluster.cc:1428] Setting key 03245198c1e38b01130ba8c5ababd058
I20260810 22:58:45.618310 13834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:45.618498 13831 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:45.618507 13834 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [term 1 LEADER]: Becoming Leader. State: Replica: aa0ca0fd423a412ba7a2b155662ab12a, State: Running, Role: LEADER
I20260810 22:58:45.618736 13834 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } }
I20260810 22:58:45.619165 13835 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } } }
I20260810 22:58:45.619254 13835 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:45.619406 13836 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa0ca0fd423a412ba7a2b155662ab12a. Latest consensus state: current_term: 1 leader_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa0ca0fd423a412ba7a2b155662ab12a" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42283 } } }
I20260810 22:58:45.619468 13836 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:45.619797 13858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:45.620369 13858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:45.622057 13858 catalog_manager.cc:1347] Generated new cluster ID: 1f830b0de8a549f79ffa5fe5d1c459ad
I20260810 22:58:45.622112 13858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:45.629947 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:45.618930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.6.47.190:42283)
0810 22:58:45.619114 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:45.619122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:45.619155 (+ 33us) server_negotiation.cc:407] Connection header received
0810 22:58:45.619288 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:45.619297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:45.619361 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:45.620115 (+ 754us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:45.620550 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.621776 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.622883 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.623481 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.623833 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:45.626127 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:45.626131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:45.626131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:45.626142 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:45.628048 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.628433 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.628437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.628437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.628568 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.628760 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.628761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.628761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.629035 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:45.629522 (+ 487us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:45.629588 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 22:58:45.629717 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260810 22:58:45.634608 13858 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:45.635332 13858 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:45.642836 13858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: Generated new TSK 0
I20260810 22:58:45.643024 13858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:45.675283 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:45.782753 13883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:45.782958 13883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:45.782981 13883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:45.784058 13883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:45.784092 13883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:45.784117 13883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:45.785589 13883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:45.785780 13883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:45.785955 13883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:45.790858 13892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:45.790891 13897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:45.790866 13894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:45.791170 13883 server_base.cc:1034] running on GCE node
I20260810 22:58:45.791493 13883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:45.791741 13883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:45.792904 13883 hybrid_clock.cc:648] HybridClock initialized: now 1786402725792899 us; error 37 us; skew 500 ppm
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:45.795598 13883 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:45.795945 13883 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:45.796020 13883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:45.796216 13883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:45.797370 13883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "ad73ac326b8a45eea1d25998d6062af3"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "8f57d358f4c0c51da922c8da4ba9e026"
server_key_iv: "216575b54f5bc013c65a708cc570d625"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.797749 13883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "ad73ac326b8a45eea1d25998d6062af3"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "8f57d358f4c0c51da922c8da4ba9e026"
server_key_iv: "216575b54f5bc013c65a708cc570d625"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.799026 13883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 22:58:45.799782 13910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.799962 13883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:45.800071 13883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "ad73ac326b8a45eea1d25998d6062af3"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "8f57d358f4c0c51da922c8da4ba9e026"
server_key_iv: "216575b54f5bc013c65a708cc570d625"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.800164 13883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:45.814818 13883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:45.817360 13883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:45.817529 13883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:45.817778 13883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:45.818115 13883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:45.818173 13883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.818218 13883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:45.818248 13883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.824656 13883 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:39413
I20260810 22:58:45.824708 14130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:39413 every 8 connection(s)
I20260810 22:58:45.825008 13883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:45.831712 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13883
I20260810 22:58:45.831832 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:45.832037 6334 external_mini_cluster.cc:1428] Setting key a57df972deeaef378308e2f06183ca0c
I20260810 22:58:45.835498 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:45.825744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:46087 (local address 127.6.47.190:42283)
0810 22:58:45.825833 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:45.825836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:45.826017 (+ 181us) server_negotiation.cc:407] Connection header received
0810 22:58:45.826501 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:45.826502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:45.826512 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:45.826566 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:45.827496 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.828068 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.829166 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.829394 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.829447 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:45.831485 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:45.831486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:45.831486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:45.831494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:45.833569 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.834283 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.834284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.834285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.834347 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.834612 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.834613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.834614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.834751 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:45.834813 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:45.835217 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 22:58:45.835332 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:58:45.835600 14137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:45.825709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42283 (local address 127.6.47.129:46087)
0810 22:58:45.825894 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:45.825930 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:45.826411 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:45.826655 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:45.826658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:45.826910 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:45.827412 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:45.827417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.828156 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.828157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:45.829090 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:45.829093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.829470 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.829471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:45.829573 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:45.830027 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:45.830034 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:45.831368 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:45.833680 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:45.833689 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:45.833691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:45.834170 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:45.834418 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:45.834418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:45.834419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:45.834553 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:45.834843 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:45.834851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:45.835037 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 22:58:45.835162 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 22:58:45.835334 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260810 22:58:45.836759 14133 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42283
I20260810 22:58:45.836860 14133 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:45.837064 14133 heartbeater.cc:511] Master 127.6.47.190:42283 requested a full tablet report, sending...
I20260810 22:58:45.837627 13765 ts_manager.cc:194] Registered new tserver with Master: ad73ac326b8a45eea1d25998d6062af3 (127.6.47.129:39413)
I20260810 22:58:45.838258 13765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:46087
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:45.859300 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:45.941309 14142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:45.941636 14142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:45.941684 14142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:45.942853 14142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:45.942927 14142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:45.942973 14142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:45.944564 14142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:45.944844 14142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:45.945088 14142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:45.950126 14150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:45.950135 14155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:45.950165 14152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:45.950367 14142 server_base.cc:1034] running on GCE node
I20260810 22:58:45.950601 14142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:45.950796 14142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:45.951931 14142 hybrid_clock.cc:648] HybridClock initialized: now 1786402725951920 us; error 32 us; skew 500 ppm
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:45.954607 14142 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:45.954910 14142 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:45.954954 14142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:45.955041 14142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:45.956022 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "7380bfa7b79e4edabf0837f5a93445d6"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "ef81c1b1ffc859115ca8a84440432a7f"
server_key_iv: "64d54f0e88b2db9b10536272ec5da774"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.956327 14142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "7380bfa7b79e4edabf0837f5a93445d6"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "ef81c1b1ffc859115ca8a84440432a7f"
server_key_iv: "64d54f0e88b2db9b10536272ec5da774"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.957599 14142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 22:58:45.958279 14168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.958453 14142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:45.958564 14142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "7380bfa7b79e4edabf0837f5a93445d6"
format_stamp: "Formatted at 2026-08-10 22:58:45 on dist-test-slave-rwrg"
server_key: "ef81c1b1ffc859115ca8a84440432a7f"
server_key_iv: "64d54f0e88b2db9b10536272ec5da774"
server_key_version: "encryptionkey@0"
I20260810 22:58:45.958642 14142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:45.972129 14142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:45.974598 14142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:45.974793 14142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:45.975030 14142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:45.975353 14142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:45.975409 14142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.975449 14142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:45.975477 14142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:45.981230 14142 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:36397
I20260810 22:58:45.981292 14373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:36397 every 8 connection(s)
I20260810 22:58:45.981600 14142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:45.984616 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14142
I20260810 22:58:45.984745 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:45.984938 6334 external_mini_cluster.cc:1428] Setting key c5abeb9bd5e2733b7682826e6a690055
Aug 10 22:58:45 dist-test-slave-rwrg krb5kdc[13693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402725, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:45.992002 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:45.982343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42283 (local address 127.6.47.130:46167)
0810 22:58:45.982513 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:45.982594 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:45.983052 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:45.983253 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:45.983256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:45.983497 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:45.983989 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:45.983994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.984693 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.984694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:45.985919 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:45.985922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.986279 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.986280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:45.986426 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:45.986922 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:45.986933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:45.988791 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:45.990660 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:45.990669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:45.990672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:45.990896 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:45.991122 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:45.991123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:45.991123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:45.991222 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:45.991528 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:45.991533 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:45.991646 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 22:58:45.991727 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 22:58:45.991844 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260810 22:58:45.992012 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:45.982367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:46167 (local address 127.6.47.190:42283)
0810 22:58:45.982437 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:45.982439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:45.982671 (+ 232us) server_negotiation.cc:407] Connection header received
0810 22:58:45.983130 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:45.983132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:45.983141 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:45.983202 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:45.984067 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.984598 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.986010 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:45.986197 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:45.986245 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:45.988888 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:45.988889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:45.988890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:45.988898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:45.990564 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.990975 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.990976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.990977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.991044 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:45.991292 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:45.991293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:45.991293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:45.991455 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:45.991536 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:45.991782 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 22:58:45.991874 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:58:45.992667 14375 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42283
I20260810 22:58:45.992784 14375 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:45.992973 14375 heartbeater.cc:511] Master 127.6.47.190:42283 requested a full tablet report, sending...
I20260810 22:58:45.993434 13765 ts_manager.cc:194] Registered new tserver with Master: 7380bfa7b79e4edabf0837f5a93445d6 (127.6.47.130:36397)
I20260810 22:58:45.993963 13765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:46167
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:46.018700 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:46.101572 14385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:46.101766 14385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:46.101789 14385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:46.102944 14385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:46.102986 14385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:46.103012 14385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:46.104657 14385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42283
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:46.104838 14385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:46.105048 14385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:46.109858 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:46.109984 14385 server_base.cc:1034] running on GCE node
W20260810 22:58:46.109898 14397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:46.110081 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:46.110359 14385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:46.110620 14385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:46.111811 14385 hybrid_clock.cc:648] HybridClock initialized: now 1786402726111806 us; error 58 us; skew 500 ppm
Aug 10 22:58:46 dist-test-slave-rwrg krb5kdc[13693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402726, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:46.114434 14385 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:46.114810 14385 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:46.114861 14385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:46.114997 14385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:46.116024 14385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "283dd35396284c708857a11ee7d82955"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "de2019904c9dc4fbb7a63286f25ec1f8"
server_key_iv: "77e8232faa48fd78389b817372685058"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.116341 14385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "283dd35396284c708857a11ee7d82955"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "de2019904c9dc4fbb7a63286f25ec1f8"
server_key_iv: "77e8232faa48fd78389b817372685058"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.117643 14385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:46.118397 14407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.118582 14385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:46.118676 14385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "283dd35396284c708857a11ee7d82955"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "de2019904c9dc4fbb7a63286f25ec1f8"
server_key_iv: "77e8232faa48fd78389b817372685058"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.118750 14385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:46.131700 14385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:46.133621 14385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:46.133764 14385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:46.133999 14385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:46.134315 14385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:46.134353 14385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.134375 14385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:46.134387 14385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.140180 14385 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:37625
I20260810 22:58:46.140221 14629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:37625 every 8 connection(s)
I20260810 22:58:46.140468 14385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:46.144447 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14385
I20260810 22:58:46.144570 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:46.144747 6334 external_mini_cluster.cc:1428] Setting key f40a33ba66b7eed19d8c18acd874ebd2
Aug 10 22:58:46 dist-test-slave-rwrg krb5kdc[13693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402726, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:46.150106 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.141153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:59583 (local address 127.6.47.190:42283)
0810 22:58:46.141193 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.141195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.141432 (+ 237us) server_negotiation.cc:407] Connection header received
0810 22:58:46.141911 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.141912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.141922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.141981 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:46.143015 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.143625 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.144769 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.145026 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.145100 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.147440 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:46.147441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:46.147442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:46.147449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:46.148724 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:46.149112 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:46.149113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:46.149113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:46.149176 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:46.149452 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:46.149453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:46.149453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:46.149577 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:46.149658 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.149933 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.150020 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:46.150305 14632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.141127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42283 (local address 127.6.47.131:59583)
0810 22:58:46.141306 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.141346 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.141830 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.142034 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.142037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.142285 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:46.142917 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.142923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.143707 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.143708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.144683 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.144686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.145138 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.145140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.145285 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.145937 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:46.145945 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:46.147321 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:46.148812 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:46.148818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:46.148820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:46.149028 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:46.149254 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:46.149255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:46.149256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:46.149379 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:46.149648 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:46.149655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:46.149792 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.149897 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.150062 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260810 22:58:46.150787 14630 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42283
I20260810 22:58:46.150882 14630 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:46.150995 14630 heartbeater.cc:511] Master 127.6.47.190:42283 requested a full tablet report, sending...
I20260810 22:58:46.151350 13765 ts_manager.cc:194] Registered new tserver with Master: 283dd35396284c708857a11ee7d82955 (127.6.47.131:37625)
I20260810 22:58:46.151678 13765 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:59583
I20260810 22:58:46.156944 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:58:46.163774 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.158154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48282 (local address 127.6.47.190:42283)
0810 22:58:46.158217 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.158219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.158334 (+ 115us) server_negotiation.cc:407] Connection header received
0810 22:58:46.158368 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.158369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.158377 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.158422 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:46.158924 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.159348 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.160030 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.160197 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.160244 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.161494 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:46.161495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:46.161495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:46.161502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:46.162796 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:46.163096 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:46.163096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:46.163096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:46.163144 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:46.163305 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:46.163305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:46.163306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:46.163429 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:46.163500 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.163609 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.163664 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:58:46.165300 13765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:46.166850 13765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:46.173512 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:39413 (local address 127.0.0.1:47214)
0810 22:58:46.169566 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.169626 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.169704 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.170034 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.170035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.170037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:46.170091 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.170093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.171450 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.171451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.172555 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.172558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173201 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.173281 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.173293 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.173371 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.173424 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":25,"thread_start_us":68,"threads_started":1}
I20260810 22:58:46.173568 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49056 (local address 127.6.47.131:37625)
0810 22:58:46.170229 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.170232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.170242 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:46.170279 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.170281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.170298 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.170376 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:46.170493 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.171495 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.172473 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173159 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173311 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.173371 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.173446 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.173485 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":87,"threads_started":1}
I20260810 22:58:46.173596 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37625 (local address 127.0.0.1:49056)
0810 22:58:46.169861 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.169870 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.169907 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.170377 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.170378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.170380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:46.170417 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.170418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.171601 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.171602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.172374 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.172376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173270 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.173381 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.173385 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.173436 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.173472 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":39,"thread_start_us":91,"threads_started":1}
I20260810 22:58:46.173759 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36397 (local address 127.0.0.1:38056)
0810 22:58:46.169713 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.169722 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.169776 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.169979 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.169983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.169985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:46.170053 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.170054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.171040 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.171041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.172204 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.172207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173273 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.173376 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.173381 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.173412 (+ 31us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.173456 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":4,"thread_start_us":251,"threads_started":1}
I20260810 22:58:46.173877 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.6.47.129:39413)
0810 22:58:46.169562 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.169567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.169799 (+ 232us) server_negotiation.cc:407] Connection header received
0810 22:58:46.169846 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.169848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.169864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.169941 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:46.170237 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.171341 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.172630 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173107 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173199 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.173260 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.173732 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.173777 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260810 22:58:46.174285 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.169355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.6.47.130:36397)
0810 22:58:46.169554 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.169559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.169793 (+ 234us) server_negotiation.cc:407] Connection header received
0810 22:58:46.169845 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.169847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.169864 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.169967 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:46.170116 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.170943 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.172304 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.173159 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.173545 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.174127 (+ 582us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.174154 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.174196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":89,"threads_started":1}
I20260810 22:58:46.174285 14497 tablet_service.cc:1467] Processing CreateTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 (DEFAULT_TABLE table=test-table [id=b7d55b11455d4ade95d8f97dd4416c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:46.174465 13993 tablet_service.cc:1467] Processing CreateTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 (DEFAULT_TABLE table=test-table [id=b7d55b11455d4ade95d8f97dd4416c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:46.174705 14497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8aa2aa4061b424e9fcb09dfb5737e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:46.174890 13993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8aa2aa4061b424e9fcb09dfb5737e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:46.174880 14236 tablet_service.cc:1467] Processing CreateTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 (DEFAULT_TABLE table=test-table [id=b7d55b11455d4ade95d8f97dd4416c83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:46.175200 14236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8aa2aa4061b424e9fcb09dfb5737e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:46.177564 14656 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Bootstrap starting.
I20260810 22:58:46.178089 14659 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Bootstrap starting.
I20260810 22:58:46.178369 14656 tablet_bootstrap.cc:654] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:46.178411 14660 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Bootstrap starting.
I20260810 22:58:46.178763 14659 tablet_bootstrap.cc:654] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:46.179010 14660 tablet_bootstrap.cc:654] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:46.179103 14656 log.cc:824] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:46.179390 14659 log.cc:824] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:46.179697 14660 log.cc:824] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:46.180199 14656 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: No bootstrap required, opened a new log
I20260810 22:58:46.180316 14656 ts_tablet_manager.cc:1397] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:46.180289 14659 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: No bootstrap required, opened a new log
I20260810 22:58:46.180449 14659 ts_tablet_manager.cc:1397] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:58:46.181181 14660 tablet_bootstrap.cc:492] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: No bootstrap required, opened a new log
I20260810 22:58:46.181257 14660 ts_tablet_manager.cc:1397] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:58:46.182175 14659 raft_consensus.cc:348] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.182348 14659 raft_consensus.cc:374] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:46.182399 14659 raft_consensus.cc:729] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 283dd35396284c708857a11ee7d82955, State: Initialized, Role: FOLLOWER
I20260810 22:58:46.182431 14656 raft_consensus.cc:348] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.182524 14659 consensus_queue.cc:260] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.182615 14656 raft_consensus.cc:374] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:46.182654 14656 raft_consensus.cc:729] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad73ac326b8a45eea1d25998d6062af3, State: Initialized, Role: FOLLOWER
I20260810 22:58:46.182783 14656 consensus_queue.cc:260] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.182888 14659 ts_tablet_manager.cc:1428] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 22:58:46.183046 14630 heartbeater.cc:503] Master 127.6.47.190:42283 was elected leader, sending a full tablet report...
I20260810 22:58:46.183077 14656 ts_tablet_manager.cc:1428] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:46.183230 14133 heartbeater.cc:503] Master 127.6.47.190:42283 was elected leader, sending a full tablet report...
I20260810 22:58:46.183743 14660 raft_consensus.cc:348] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.183946 14660 raft_consensus.cc:374] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:46.183990 14660 raft_consensus.cc:729] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7380bfa7b79e4edabf0837f5a93445d6, State: Initialized, Role: FOLLOWER
I20260810 22:58:46.184108 14660 consensus_queue.cc:260] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.184453 14660 ts_tablet_manager.cc:1428] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:46.184538 14375 heartbeater.cc:503] Master 127.6.47.190:42283 was elected leader, sending a full tablet report...
W20260810 22:58:46.232156 14376 tablet.cc:2369] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:46.325735 14134 tablet.cc:2369] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:46.391041 14631 tablet.cc:2369] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:46.470249 14672 raft_consensus.cc:482] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:46.470420 14672 raft_consensus.cc:504] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.470829 14672 leader_election.cc:288] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad73ac326b8a45eea1d25998d6062af3 (127.6.47.129:39413), 283dd35396284c708857a11ee7d82955 (127.6.47.131:37625)
I20260810 22:58:46.473685 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.471012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:39413 (local address 127.6.47.130:47283)
0810 22:58:46.471078 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.471091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.471135 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.471306 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.471307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.471310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:46.471389 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.471392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.472059 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.472060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.472868 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.472870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.473437 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.473439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.473520 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.473525 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.473581 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.473629 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 22:58:46.473711 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.471087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:47283 (local address 127.6.47.129:39413)
0810 22:58:46.471163 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.471165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.471175 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:46.471207 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.471208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.471221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.471281 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:46.471474 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.471985 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.472959 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.473362 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.473419 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.473426 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.473598 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.473633 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 22:58:46.473881 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.471044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37625 (local address 127.6.47.130:34709)
0810 22:58:46.471234 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.471242 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.471280 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.471419 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.471420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.471421 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:46.471520 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.471522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.472151 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.472152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.472983 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:46.472985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.473610 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.473611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:46.473680 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.473684 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.473745 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.473785 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":10,"thread_start_us":61,"threads_started":1}
I20260810 22:58:46.473891 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.471214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:34709 (local address 127.6.47.131:37625)
0810 22:58:46.471283 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.471286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.471297 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:46.471325 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.471326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.471336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.471405 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:46.471598 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.472059 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.473076 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.473486 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.473564 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.473573 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.473772 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.473816 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:46.474098 14050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8aa2aa4061b424e9fcb09dfb5737e33" candidate_uuid: "7380bfa7b79e4edabf0837f5a93445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad73ac326b8a45eea1d25998d6062af3" is_pre_election: true
I20260810 22:58:46.474309 14050 raft_consensus.cc:2454] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380bfa7b79e4edabf0837f5a93445d6 in term 0.
I20260810 22:58:46.474308 14538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8aa2aa4061b424e9fcb09dfb5737e33" candidate_uuid: "7380bfa7b79e4edabf0837f5a93445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283dd35396284c708857a11ee7d82955" is_pre_election: true
I20260810 22:58:46.474469 14538 raft_consensus.cc:2454] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380bfa7b79e4edabf0837f5a93445d6 in term 0.
I20260810 22:58:46.474597 14170 leader_election.cc:302] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7380bfa7b79e4edabf0837f5a93445d6, ad73ac326b8a45eea1d25998d6062af3; no voters:
I20260810 22:58:46.474723 14672 raft_consensus.cc:2781] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:46.474778 14672 raft_consensus.cc:482] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:46.474795 14672 raft_consensus.cc:3037] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:46.475485 14672 raft_consensus.cc:504] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.475625 14672 leader_election.cc:288] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [CANDIDATE]: Term 1 election: Requested vote from peers ad73ac326b8a45eea1d25998d6062af3 (127.6.47.129:39413), 283dd35396284c708857a11ee7d82955 (127.6.47.131:37625)
I20260810 22:58:46.475780 14050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8aa2aa4061b424e9fcb09dfb5737e33" candidate_uuid: "7380bfa7b79e4edabf0837f5a93445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad73ac326b8a45eea1d25998d6062af3"
I20260810 22:58:46.475793 14538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8aa2aa4061b424e9fcb09dfb5737e33" candidate_uuid: "7380bfa7b79e4edabf0837f5a93445d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "283dd35396284c708857a11ee7d82955"
I20260810 22:58:46.475864 14050 raft_consensus.cc:3037] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:46.475864 14538 raft_consensus.cc:3037] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:46.476781 14538 raft_consensus.cc:2454] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380bfa7b79e4edabf0837f5a93445d6 in term 1.
I20260810 22:58:46.476781 14050 raft_consensus.cc:2454] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380bfa7b79e4edabf0837f5a93445d6 in term 1.
I20260810 22:58:46.476972 14170 leader_election.cc:302] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7380bfa7b79e4edabf0837f5a93445d6, ad73ac326b8a45eea1d25998d6062af3; no voters:
I20260810 22:58:46.477108 14672 raft_consensus.cc:2781] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:46.477285 14672 raft_consensus.cc:686] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 1 LEADER]: Becoming Leader. State: Replica: 7380bfa7b79e4edabf0837f5a93445d6, State: Running, Role: LEADER
I20260810 22:58:46.477386 14672 consensus_queue.cc:237] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } }
I20260810 22:58:46.478220 13765 catalog_manager.cc:5495] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7380bfa7b79e4edabf0837f5a93445d6 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "7380bfa7b79e4edabf0837f5a93445d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7380bfa7b79e4edabf0837f5a93445d6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:46.527832 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.525660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.6.47.130:36397)
0810 22:58:46.525740 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.525742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.525749 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:46.525785 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.525786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.525806 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.525867 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:46.525980 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.526428 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.527120 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:46.527289 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:46.527337 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:46.527477 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:46.527575 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:46.527639 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.527671 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.527686 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 22:58:46.529860 14538 raft_consensus.cc:1261] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 1 FOLLOWER]: Refusing update from remote peer 7380bfa7b79e4edabf0837f5a93445d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:46.529860 14050 raft_consensus.cc:1261] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 1 FOLLOWER]: Refusing update from remote peer 7380bfa7b79e4edabf0837f5a93445d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:46.530267 14672 consensus_queue.cc:1035] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "283dd35396284c708857a11ee7d82955" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:46.530411 14677 consensus_queue.cc:1035] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad73ac326b8a45eea1d25998d6062af3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 39413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:46.533133 14689 mvcc.cc:205] Tried to move back new op lower bound from 7317105567863316480 to 7317105567652102144. Current Snapshot: MvccSnapshot[applied={T|T < 7317105567863316480}]
I20260810 22:58:46.534039 14687 mvcc.cc:205] Tried to move back new op lower bound from 7317105567863316480 to 7317105567652102144. Current Snapshot: MvccSnapshot[applied={T|T < 7317105567863316480}]
I20260810 22:58:46.535248 13765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:46.535351 13765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:46.536120 13765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa0ca0fd423a412ba7a2b155662ab12a: Sending DeleteTablet for 3 replicas of tablet e8aa2aa4061b424e9fcb09dfb5737e33
I20260810 22:58:46.536540 13993 tablet_service.cc:1514] Processing DeleteTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:46 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:47214
I20260810 22:58:46.536536 14236 tablet_service.cc:1514] Processing DeleteTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:46 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:38056
I20260810 22:58:46.536556 14497 tablet_service.cc:1514] Processing DeleteTablet for tablet e8aa2aa4061b424e9fcb09dfb5737e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:46 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:49056
I20260810 22:58:46.536794 14708 tablet_replica.cc:330] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3: stopping tablet replica
I20260810 22:58:46.536870 14706 tablet_replica.cc:330] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: stopping tablet replica
I20260810 22:58:46.536897 14708 raft_consensus.cc:2229] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:46.536966 14706 raft_consensus.cc:2229] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:46.537001 14708 raft_consensus.cc:2258] T e8aa2aa4061b424e9fcb09dfb5737e33 P ad73ac326b8a45eea1d25998d6062af3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:46.537061 14706 raft_consensus.cc:2258] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:46.537226 14707 tablet_replica.cc:330] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: stopping tablet replica
I20260810 22:58:46.537245 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13883
I20260810 22:58:46.537322 14707 raft_consensus.cc:2229] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:46.537425 14707 raft_consensus.cc:2258] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:46.541749 14707 ts_tablet_manager.cc:1905] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:46.544134 14707 ts_tablet_manager.cc:1918] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:46.544222 14707 log.cc:1196] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/e8aa2aa4061b424e9fcb09dfb5737e33
I20260810 22:58:46.544518 14707 ts_tablet_manager.cc:1939] T e8aa2aa4061b424e9fcb09dfb5737e33 P 7380bfa7b79e4edabf0837f5a93445d6: Deleting consensus metadata
I20260810 22:58:46.544992 13737 catalog_manager.cc:4841] TS 7380bfa7b79e4edabf0837f5a93445d6 (127.6.47.130:36397): tablet e8aa2aa4061b424e9fcb09dfb5737e33 (table test-table [id=b7d55b11455d4ade95d8f97dd4416c83]) successfully deleted
I20260810 22:58:46.546936 14706 ts_tablet_manager.cc:1905] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:46.548635 14706 ts_tablet_manager.cc:1918] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:46.548727 14706 log.cc:1196] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/e8aa2aa4061b424e9fcb09dfb5737e33
I20260810 22:58:46.549019 14706 ts_tablet_manager.cc:1939] T e8aa2aa4061b424e9fcb09dfb5737e33 P 283dd35396284c708857a11ee7d82955: Deleting consensus metadata
I20260810 22:58:46.549515 13737 catalog_manager.cc:4841] TS 283dd35396284c708857a11ee7d82955 (127.6.47.131:37625): tablet e8aa2aa4061b424e9fcb09dfb5737e33 (table test-table [id=b7d55b11455d4ade95d8f97dd4416c83]) successfully deleted
W20260810 22:58:46.551311 13732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:39413 (error 108)
I20260810 22:58:46.551541 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14142
I20260810 22:58:46.558591 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.557579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:39413 (local address 127.0.0.1:47216)
0810 22:58:46.558504 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.558551 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:39413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":903}
W20260810 22:58:46.558725 13732 catalog_manager.cc:4584] TS ad73ac326b8a45eea1d25998d6062af3 (127.6.47.129:39413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8aa2aa4061b424e9fcb09dfb5737e33: Network error: Client connection negotiation failed: client connection to 127.6.47.129:39413: connect: Connection refused (error 111)
I20260810 22:58:46.572558 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14385
I20260810 22:58:46.585640 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13711
2026-08-10T22:58:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2803 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T22:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:46Z Disabled control of system clock
I20260810 22:58:46.623347 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37367
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:37367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:58:46.705334 14717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:46.705551 14717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:46.705574 14717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:46.706729 14717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:46.706772 14717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:46.706785 14717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:46.706797 14717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:46.706809 14717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:46.708242 14717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:37367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:46.708549 14717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:46.708875 14717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:46.713862 14734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:46.713886 14731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:46.713984 14717 server_base.cc:1034] running on GCE node
W20260810 22:58:46.713891 14732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:46.714480 14717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:46.714859 14717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:46.716015 14717 hybrid_clock.cc:648] HybridClock initialized: now 1786402726716000 us; error 44 us; skew 500 ppm
I20260810 22:58:46.717363 14717 webserver.cc:460] Webserver started at http://127.6.47.190:33837/ using document root <none> and password file <none>
I20260810 22:58:46.717648 14717 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:46.717720 14717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:46.717876 14717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:46.719015 14717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "1ef7311f18de4286bd890c364e52ce86"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "288ebcbd5082907527bd188f519e0542"
server_key_iv: "fe677b1419e984e44fb7e86e814f786c"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.719416 14717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "1ef7311f18de4286bd890c364e52ce86"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "288ebcbd5082907527bd188f519e0542"
server_key_iv: "fe677b1419e984e44fb7e86e814f786c"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.720788 14717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:46.721520 14743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.721688 14717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:46.721798 14717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "1ef7311f18de4286bd890c364e52ce86"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "288ebcbd5082907527bd188f519e0542"
server_key_iv: "fe677b1419e984e44fb7e86e814f786c"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.721889 14717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:46.725771 14717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:46.726058 14717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:46.726217 14717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:46.730397 14717 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:37367
I20260810 22:58:46.730432 14852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:37367 every 8 connection(s)
I20260810 22:58:46.730774 14717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:46.731485 14855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:46.734354 14855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: Bootstrap starting.
I20260810 22:58:46.734946 14855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:46.735268 14855 log.cc:824] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:46.736003 14855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: No bootstrap required, opened a new log
I20260810 22:58:46.737735 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14717
I20260810 22:58:46.737841 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:46.738025 6334 external_mini_cluster.cc:1428] Setting key 02a496977aa8ba5f0d9732a57bb42f68
I20260810 22:58:46.739269 14855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } }
I20260810 22:58:46.739445 14855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:46.739485 14855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef7311f18de4286bd890c364e52ce86, State: Initialized, Role: FOLLOWER
I20260810 22:58:46.739620 14855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } }
I20260810 22:58:46.739701 14855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:46.739732 14855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:46.739799 14855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:46.740687 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.738546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.6.47.190:37367)
0810 22:58:46.738781 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.738792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.738855 (+ 63us) server_negotiation.cc:407] Connection header received
0810 22:58:46.739230 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.739236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.739273 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.739408 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:46.740252 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:46.740255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:46.740256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:46.740258 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:46.740284 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:46.740293 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:46.740375 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.740449 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.740531 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260810 22:58:46.810529 14855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } }
I20260810 22:58:46.810791 14855 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef7311f18de4286bd890c364e52ce86; no voters:
I20260810 22:58:46.811065 14855 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:46.811158 14861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:46.811354 14861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef7311f18de4286bd890c364e52ce86, State: Running, Role: LEADER
I20260810 22:58:46.811419 14855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:46.811463 14861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } }
I20260810 22:58:46.811842 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef7311f18de4286bd890c364e52ce86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } } }
I20260810 22:58:46.811949 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:46.812140 14863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef7311f18de4286bd890c364e52ce86. Latest consensus state: current_term: 1 leader_uuid: "1ef7311f18de4286bd890c364e52ce86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef7311f18de4286bd890c364e52ce86" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 37367 } } }
I20260810 22:58:46.812216 14863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:46.812513 14867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:46.813220 14867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:46.815235 14867 catalog_manager.cc:1347] Generated new cluster ID: c628b1dc26914202be71306df9760229
I20260810 22:58:46.815294 14867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:46.823567 14867 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:46.823740 14867 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:46.828727 14867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: Generated new TSK 0
I20260810 22:58:46.828908 14867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:58:46.834151 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--builtin_ntp_servers=127.6.47.148:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:58:46.937180 14902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:46.937426 14902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:46.937469 14902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:46.938690 14902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:46.938756 14902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:46.938800 14902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:46.940168 14902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:46.940446 14902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:46.940840 14902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:46.949229 14912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:46.949256 14910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:46.949568 14915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:46.950068 14902 server_base.cc:1034] running on GCE node
I20260810 22:58:46.950294 14902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:46.950563 14902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:46.951728 14902 hybrid_clock.cc:648] HybridClock initialized: now 1786402726951718 us; error 28 us; skew 500 ppm
I20260810 22:58:46.953202 14902 webserver.cc:460] Webserver started at http://127.6.47.129:38895/ using document root <none> and password file <none>
I20260810 22:58:46.953449 14902 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:46.953519 14902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:46.953660 14902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:46.954769 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "506082469df84a7780a3064d7d5f7b03"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "d90c4f0c83a324fb03f76be359b58de7"
server_key_iv: "94f65f9419c7a59e790eb831d8de90f3"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.955148 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "506082469df84a7780a3064d7d5f7b03"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "d90c4f0c83a324fb03f76be359b58de7"
server_key_iv: "94f65f9419c7a59e790eb831d8de90f3"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.956554 14902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 22:58:46.957460 14929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.957667 14902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:46.957768 14902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "506082469df84a7780a3064d7d5f7b03"
format_stamp: "Formatted at 2026-08-10 22:58:46 on dist-test-slave-rwrg"
server_key: "d90c4f0c83a324fb03f76be359b58de7"
server_key_iv: "94f65f9419c7a59e790eb831d8de90f3"
server_key_version: "encryptionkey@0"
I20260810 22:58:46.957861 14902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:46.961199 14902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:46.961508 14902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:46.961665 14902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:46.961897 14902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:46.962229 14902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:46.962287 14902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.962327 14902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:46.962353 14902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:46.968279 14902 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:44643
I20260810 22:58:46.968743 15148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:44643 every 8 connection(s)
I20260810 22:58:46.968793 14902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:46.972343 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.969641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:56787 (local address 127.6.47.190:37367)
0810 22:58:46.969694 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:46.969696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:46.969977 (+ 281us) server_negotiation.cc:407] Connection header received
0810 22:58:46.970585 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:46.970586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:46.970594 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:46.970637 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:46.972114 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:46.972115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:46.972115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:46.972119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:46.972134 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:46.972137 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:46.972199 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:46.972276 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 22:58:46.972291 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 22:58:46.972735 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:46.969636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:37367 (local address 127.6.47.129:56787)
0810 22:58:46.969828 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:46.969873 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:46.970470 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:46.970710 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:46.970714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:46.970725 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:46.971936 (+ 1211us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:46.971937 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:46.971977 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:46.972015 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:46.972200 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:46.972204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:46.972207 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:46.972278 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 22:58:46.972340 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260810 22:58:46.973238 15149 heartbeater.cc:347] Connected to a master server at 127.6.47.190:37367
I20260810 22:58:46.973325 15149 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:46.973387 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14902
I20260810 22:58:46.973464 15149 heartbeater.cc:511] Master 127.6.47.190:37367 requested a full tablet report, sending...
I20260810 22:58:46.973472 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:46.973649 6334 external_mini_cluster.cc:1428] Setting key f3266526a9890ed129dd41c9739fa7cd
I20260810 22:58:46.973977 14779 ts_manager.cc:194] Registered new tserver with Master: 506082469df84a7780a3064d7d5f7b03 (127.6.47.129:44643)
I20260810 22:58:46.974754 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--builtin_ntp_servers=127.6.47.148:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:58:47.061785 15154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:47.061995 15154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:47.062018 15154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:47.063202 15154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:47.063244 15154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:47.063270 15154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:47.064738 15154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:47.064991 15154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:47.065304 15154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:47.070564 15163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.070570 15160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:47.070654 15154 server_base.cc:1034] running on GCE node
W20260810 22:58:47.070573 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:47.070981 15154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:47.071273 15154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:47.072433 15154 hybrid_clock.cc:648] HybridClock initialized: now 1786402727072422 us; error 41 us; skew 500 ppm
I20260810 22:58:47.073834 15154 webserver.cc:460] Webserver started at http://127.6.47.130:41311/ using document root <none> and password file <none>
I20260810 22:58:47.074141 15154 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:47.074193 15154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:47.074591 15154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:47.075742 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "9de1205f185640b6839552c68e9c6ad6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "833c7235f65a46b4a13f4480212a4d96"
server_key_iv: "1a015fbded43f0e6f3d8674e2caeb159"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.076171 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "9de1205f185640b6839552c68e9c6ad6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "833c7235f65a46b4a13f4480212a4d96"
server_key_iv: "1a015fbded43f0e6f3d8674e2caeb159"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.077641 15154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:47.078416 15183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.078594 15154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:47.078739 15154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "9de1205f185640b6839552c68e9c6ad6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "833c7235f65a46b4a13f4480212a4d96"
server_key_iv: "1a015fbded43f0e6f3d8674e2caeb159"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.078819 15154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:47.082604 15154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:47.082890 15154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:47.083037 15154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:47.083254 15154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:47.083585 15154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:47.083643 15154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.083685 15154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:47.083712 15154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.090253 15154 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:41171
I20260810 22:58:47.090306 15414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:41171 every 8 connection(s)
I20260810 22:58:47.090659 15154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:47.093878 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.091419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:44445 (local address 127.6.47.190:37367)
0810 22:58:47.091491 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.091494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.091763 (+ 269us) server_negotiation.cc:407] Connection header received
0810 22:58:47.092287 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.092289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.092296 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.092351 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.093573 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.093575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.093575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.093580 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.093598 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.093602 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.093702 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.093788 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.093804 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:58:47.094092 15420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.091395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:37367 (local address 127.6.47.130:44445)
0810 22:58:47.091628 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.091664 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.092173 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.092437 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.092442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.092452 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.093405 (+ 953us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.093405 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.093434 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.093463 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.093700 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.093708 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.093712 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.093800 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.093866 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260810 22:58:47.094592 15417 heartbeater.cc:347] Connected to a master server at 127.6.47.190:37367
I20260810 22:58:47.094699 15417 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:47.094822 15417 heartbeater.cc:511] Master 127.6.47.190:37367 requested a full tablet report, sending...
I20260810 22:58:47.095193 14778 ts_manager.cc:194] Registered new tserver with Master: 9de1205f185640b6839552c68e9c6ad6 (127.6.47.130:41171)
I20260810 22:58:47.100741 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 15154
I20260810 22:58:47.100857 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:47.101049 6334 external_mini_cluster.cc:1428] Setting key a916581fdc706c9e8b156eaa0b0067bc
I20260810 22:58:47.102061 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--builtin_ntp_servers=127.6.47.148:41479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 22:58:47.195791 15423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:47.195991 15423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:47.196012 15423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:47.197539 15423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:47.197585 15423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:47.197611 15423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:47.199113 15423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:37367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:47.199393 15423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:47.199697 15423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:47.204877 15432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.204916 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.204855 15428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:47.205189 15423 server_base.cc:1034] running on GCE node
I20260810 22:58:47.205466 15423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:47.205718 15423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:47.206884 15423 hybrid_clock.cc:648] HybridClock initialized: now 1786402727206867 us; error 28 us; skew 500 ppm
I20260810 22:58:47.208117 15423 webserver.cc:460] Webserver started at http://127.6.47.131:38411/ using document root <none> and password file <none>
I20260810 22:58:47.208376 15423 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:47.208451 15423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:47.208624 15423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:47.209735 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "2f7df9e28e3742afaeb062a20155c381"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "0e1f681ea1900b41a33be6dc5da4f3db"
server_key_iv: "a506bb8aafa26a8be3170f371c3cdb4b"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.210114 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "2f7df9e28e3742afaeb062a20155c381"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "0e1f681ea1900b41a33be6dc5da4f3db"
server_key_iv: "a506bb8aafa26a8be3170f371c3cdb4b"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.211478 15423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:47.212828 15447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.213007 15423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.213122 15423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "2f7df9e28e3742afaeb062a20155c381"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "0e1f681ea1900b41a33be6dc5da4f3db"
server_key_iv: "a506bb8aafa26a8be3170f371c3cdb4b"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.213212 15423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:47.216569 15423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:47.216847 15423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:47.216995 15423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:47.217299 15423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:47.217676 15423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:47.217739 15423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.217782 15423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:47.217810 15423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.224695 15423 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:39569
I20260810 22:58:47.224839 15670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:39569 every 8 connection(s)
I20260810 22:58:47.225132 15423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:47.227821 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 15423
I20260810 22:58:47.227941 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:47.228138 6334 external_mini_cluster.cc:1428] Setting key 243542348bba216b8911ccf6778ed9f1
I20260810 22:58:47.229034 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.225894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:55863 (local address 127.6.47.190:37367)
0810 22:58:47.225948 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.225950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.226316 (+ 366us) server_negotiation.cc:407] Connection header received
0810 22:58:47.226747 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.226749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.226756 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.226835 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.228711 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.228712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.228712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.228716 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.228733 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.228737 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.228796 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.228938 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.228954 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 22:58:47.229228 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.225884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:37367 (local address 127.6.47.131:55863)
0810 22:58:47.226193 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.226223 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.226641 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.226893 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.226896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.226904 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.228135 (+ 1231us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.228136 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.228176 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.228215 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.228849 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.228855 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.228858 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.228926 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.228975 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":82,"threads_started":1}
I20260810 22:58:47.229668 15672 heartbeater.cc:347] Connected to a master server at 127.6.47.190:37367
I20260810 22:58:47.229773 15672 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:47.229935 15672 heartbeater.cc:511] Master 127.6.47.190:37367 requested a full tablet report, sending...
I20260810 22:58:47.230280 14778 ts_manager.cc:194] Registered new tserver with Master: 2f7df9e28e3742afaeb062a20155c381 (127.6.47.131:39569)
I20260810 22:58:47.239984 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:58:47.242548 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.241476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.6.47.190:37367)
0810 22:58:47.241563 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.241565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.241642 (+ 77us) server_negotiation.cc:407] Connection header received
0810 22:58:47.241747 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.241748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.241754 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.241811 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.242335 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.242336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.242337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.242341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.242355 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.242359 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.242407 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.242459 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.242475 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:58:47.244030 14778 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:47.245658 14778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:47.249610 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.247908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:44643 (local address 127.0.0.1:49336)
0810 22:58:47.248248 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.248283 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.248337 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.248627 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.248635 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.248641 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.249139 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.249140 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.249157 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.249182 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.249466 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.249473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.249477 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.249529 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.249542 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":69,"threads_started":1}
I20260810 22:58:47.249742 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.247905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:41171 (local address 127.0.0.1:54682)
0810 22:58:47.248230 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.248271 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.248340 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.248968 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.248969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.248971 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.249320 (+ 349us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.249321 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.249328 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.249341 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.249583 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.249584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.249585 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.249630 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.249642 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":67,"threads_started":1}
I20260810 22:58:47.249961 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.248027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49336 (local address 127.6.47.129:44643)
0810 22:58:47.248468 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.248473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.248484 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:47.248530 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.248532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.248543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.248629 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.249289 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.249298 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.249298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.249303 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.249350 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.249380 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.249826 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.249853 (+ 27us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.249870 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":90,"threads_started":1}
I20260810 22:58:47.249987 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.248106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.6.47.130:41171)
0810 22:58:47.248529 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.248533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.248541 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:47.248566 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.248570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.248579 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.248646 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.249422 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.249430 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.249430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.249435 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.249469 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.249497 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.249555 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.249885 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.249902 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":71,"threads_started":1}
I20260810 22:58:47.250204 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.248145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:39569 (local address 127.0.0.1:50062)
0810 22:58:47.249128 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.249136 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.249205 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.249359 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.249360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.249363 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.249825 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.249825 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.249833 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.249847 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.250077 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.250078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.250079 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.250127 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.250141 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":55,"threads_started":1}
I20260810 22:58:47.250502 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.248280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.6.47.131:39569)
0810 22:58:47.248483 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.248507 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.249209 (+ 702us) server_negotiation.cc:407] Connection header received
0810 22:58:47.249267 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.249270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.249279 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.249355 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.249930 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.249937 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.249937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.249941 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.249973 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.249996 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.250051 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.250297 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.250313 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260810 22:58:47.250721 15015 tablet_service.cc:1467] Processing CreateTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 (DEFAULT_TABLE table=test-table [id=343a2bef3a70405db4124b9fd3dcacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:47.250735 15275 tablet_service.cc:1467] Processing CreateTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 (DEFAULT_TABLE table=test-table [id=343a2bef3a70405db4124b9fd3dcacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:47.251075 15015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:47.251080 15275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:47.253098 15543 tablet_service.cc:1467] Processing CreateTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 (DEFAULT_TABLE table=test-table [id=343a2bef3a70405db4124b9fd3dcacd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:47.253432 15543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:47.254017 15692 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: Bootstrap starting.
I20260810 22:58:47.254204 15693 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Bootstrap starting.
I20260810 22:58:47.254738 15692 tablet_bootstrap.cc:654] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:47.254869 15693 tablet_bootstrap.cc:654] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:47.255205 15693 log.cc:824] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:47.255239 15692 log.cc:824] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:47.255981 15693 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: No bootstrap required, opened a new log
I20260810 22:58:47.256050 15693 ts_tablet_manager.cc:1397] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:47.256863 15698 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Bootstrap starting.
I20260810 22:58:47.257505 15698 tablet_bootstrap.cc:654] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:47.257827 15698 log.cc:824] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:47.258649 15698 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: No bootstrap required, opened a new log
I20260810 22:58:47.258719 15698 ts_tablet_manager.cc:1397] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:47.258913 15693 raft_consensus.cc:348] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.259105 15693 raft_consensus.cc:374] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:47.259142 15693 raft_consensus.cc:729] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de1205f185640b6839552c68e9c6ad6, State: Initialized, Role: FOLLOWER
I20260810 22:58:47.259264 15693 consensus_queue.cc:260] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.259546 15693 ts_tablet_manager.cc:1428] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260810 22:58:47.259577 15417 heartbeater.cc:503] Master 127.6.47.190:37367 was elected leader, sending a full tablet report...
I20260810 22:58:47.260190 15692 tablet_bootstrap.cc:492] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: No bootstrap required, opened a new log
I20260810 22:58:47.260309 15692 ts_tablet_manager.cc:1397] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 22:58:47.261415 15698 raft_consensus.cc:348] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.261538 15698 raft_consensus.cc:374] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:47.261560 15698 raft_consensus.cc:729] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f7df9e28e3742afaeb062a20155c381, State: Initialized, Role: FOLLOWER
I20260810 22:58:47.261686 15698 consensus_queue.cc:260] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.261945 15698 ts_tablet_manager.cc:1428] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 22:58:47.261991 15672 heartbeater.cc:503] Master 127.6.47.190:37367 was elected leader, sending a full tablet report...
I20260810 22:58:47.263199 15692 raft_consensus.cc:348] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.263335 15692 raft_consensus.cc:374] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:47.263358 15692 raft_consensus.cc:729] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506082469df84a7780a3064d7d5f7b03, State: Initialized, Role: FOLLOWER
I20260810 22:58:47.263459 15692 consensus_queue.cc:260] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.263695 15692 ts_tablet_manager.cc:1428] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:47.263775 15149 heartbeater.cc:503] Master 127.6.47.190:37367 was elected leader, sending a full tablet report...
I20260810 22:58:47.301815 15707 raft_consensus.cc:482] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:47.301996 15707 raft_consensus.cc:504] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.302400 15707 leader_election.cc:288] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f7df9e28e3742afaeb062a20155c381 (127.6.47.131:39569), 9de1205f185640b6839552c68e9c6ad6 (127.6.47.130:41171)
I20260810 22:58:47.303836 15689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.302708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50607 (local address 127.6.47.130:41171)
0810 22:58:47.302754 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.302756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.302914 (+ 158us) server_negotiation.cc:407] Connection header received
0810 22:58:47.302944 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.302945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.302950 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.303002 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.303602 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.303604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.303604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.303608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.303625 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.303629 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.303688 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.303746 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.303762 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:47.303836 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.302705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:39389 (local address 127.6.47.131:39569)
0810 22:58:47.302754 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.302756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.302766 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:47.302796 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.302797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.302803 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.302872 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.303540 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.303541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.303541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.303563 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.303584 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.303590 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.303654 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.303748 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.303762 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 22:58:47.303866 15153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.302613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:39569 (local address 127.6.47.129:39389)
0810 22:58:47.302675 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.302688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.302729 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.302905 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.302907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.302910 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.303396 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.303396 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.303404 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.303433 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.303676 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.303677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.303678 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.303736 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.303766 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":2}
I20260810 22:58:47.303882 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.302630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:41171 (local address 127.6.47.129:50607)
0810 22:58:47.302815 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.302830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.302891 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.303030 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.303031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.303034 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:47.303502 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:58:47.303503 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:47.303508 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:58:47.303517 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:47.303694 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:47.303695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:47.303695 (+ 0us) client_negotiation.cc:769] Sending connection context
0810 22:58:47.303749 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 22:58:47.303764 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":131,"threads_started":1}
I20260810 22:58:47.304267 15583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365028b3e3ea4b0dbcf5e18c81a3edc7" candidate_uuid: "506082469df84a7780a3064d7d5f7b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7df9e28e3742afaeb062a20155c381" is_pre_election: true
I20260810 22:58:47.304373 15314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365028b3e3ea4b0dbcf5e18c81a3edc7" candidate_uuid: "506082469df84a7780a3064d7d5f7b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de1205f185640b6839552c68e9c6ad6" is_pre_election: true
I20260810 22:58:47.304431 15583 raft_consensus.cc:2454] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506082469df84a7780a3064d7d5f7b03 in term 0.
I20260810 22:58:47.304524 15314 raft_consensus.cc:2454] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506082469df84a7780a3064d7d5f7b03 in term 0.
I20260810 22:58:47.304700 14933 leader_election.cc:302] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f7df9e28e3742afaeb062a20155c381, 506082469df84a7780a3064d7d5f7b03; no voters:
I20260810 22:58:47.304837 15707 raft_consensus.cc:2781] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:47.304898 15707 raft_consensus.cc:482] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:47.304926 15707 raft_consensus.cc:3037] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:47.328008 15707 raft_consensus.cc:504] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.328262 15707 leader_election.cc:288] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [CANDIDATE]: Term 1 election: Requested vote from peers 2f7df9e28e3742afaeb062a20155c381 (127.6.47.131:39569), 9de1205f185640b6839552c68e9c6ad6 (127.6.47.130:41171)
I20260810 22:58:47.328543 15314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365028b3e3ea4b0dbcf5e18c81a3edc7" candidate_uuid: "506082469df84a7780a3064d7d5f7b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de1205f185640b6839552c68e9c6ad6"
I20260810 22:58:47.328558 15583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365028b3e3ea4b0dbcf5e18c81a3edc7" candidate_uuid: "506082469df84a7780a3064d7d5f7b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f7df9e28e3742afaeb062a20155c381"
I20260810 22:58:47.328683 15314 raft_consensus.cc:3037] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:47.328706 15583 raft_consensus.cc:3037] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:47.329614 15314 raft_consensus.cc:2454] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506082469df84a7780a3064d7d5f7b03 in term 1.
I20260810 22:58:47.329881 14931 leader_election.cc:302] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506082469df84a7780a3064d7d5f7b03, 9de1205f185640b6839552c68e9c6ad6; no voters:
I20260810 22:58:47.329921 15583 raft_consensus.cc:2454] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506082469df84a7780a3064d7d5f7b03 in term 1.
I20260810 22:58:47.329999 15707 raft_consensus.cc:2781] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:47.330161 15707 raft_consensus.cc:686] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [term 1 LEADER]: Becoming Leader. State: Replica: 506082469df84a7780a3064d7d5f7b03, State: Running, Role: LEADER
I20260810 22:58:47.330308 15707 consensus_queue.cc:237] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } }
I20260810 22:58:47.331233 14778 catalog_manager.cc:5495] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 506082469df84a7780a3064d7d5f7b03 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "506082469df84a7780a3064d7d5f7b03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506082469df84a7780a3064d7d5f7b03" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 44643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 } health_report { overall_health: UNKNOWN } } }
W20260810 22:58:47.341251 15418 tablet.cc:2369] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:47.343653 15688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.342649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49342 (local address 127.6.47.129:44643)
0810 22:58:47.342724 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.342726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.342736 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:47.342782 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.342783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.342796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.342845 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.343447 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.343448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.343448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.343452 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.343465 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.343469 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.343514 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.343544 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.343555 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 22:58:47.345585 15314 raft_consensus.cc:1261] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 1 FOLLOWER]: Refusing update from remote peer 506082469df84a7780a3064d7d5f7b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:47.345597 15583 raft_consensus.cc:1261] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 1 FOLLOWER]: Refusing update from remote peer 506082469df84a7780a3064d7d5f7b03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:47.346000 15715 consensus_queue.cc:1035] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f7df9e28e3742afaeb062a20155c381" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:47.346153 15707 consensus_queue.cc:1035] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 506082469df84a7780a3064d7d5f7b03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9de1205f185640b6839552c68e9c6ad6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:47.348759 15725 mvcc.cc:205] Tried to move back new op lower bound from 7317105571204845568 to 7317105571145740288. Current Snapshot: MvccSnapshot[applied={T|T < 7317105571204845568}]
I20260810 22:58:47.350097 15724 mvcc.cc:205] Tried to move back new op lower bound from 7317105571204845568 to 7317105571145740288. Current Snapshot: MvccSnapshot[applied={T|T < 7317105571204845568}]
I20260810 22:58:47.351004 14779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:47.351100 14779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:47.351881 14779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ef7311f18de4286bd890c364e52ce86: Sending DeleteTablet for 3 replicas of tablet 365028b3e3ea4b0dbcf5e18c81a3edc7
I20260810 22:58:47.352207 15015 tablet_service.cc:1514] Processing DeleteTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:47 UTC) from {username='slave'} at 127.0.0.1:49336
I20260810 22:58:47.352241 15275 tablet_service.cc:1514] Processing DeleteTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:47 UTC) from {username='slave'} at 127.0.0.1:54682
I20260810 22:58:47.352464 15543 tablet_service.cc:1514] Processing DeleteTablet for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:47 UTC) from {username='slave'} at 127.0.0.1:50062
I20260810 22:58:47.352598 15751 tablet_replica.cc:330] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: stopping tablet replica
I20260810 22:58:47.352723 15751 raft_consensus.cc:2229] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:47.352807 15752 tablet_replica.cc:330] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: stopping tablet replica
I20260810 22:58:47.352826 15751 raft_consensus.cc:2258] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:47.352871 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14902
I20260810 22:58:47.352995 15752 raft_consensus.cc:2229] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:47.353152 15752 raft_consensus.cc:2258] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:47.353648 15751 ts_tablet_manager.cc:1905] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 9de1205f185640b6839552c68e9c6ad6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:47.354167 15752 ts_tablet_manager.cc:1905] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 22:58:47.369976 14749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:44643 (error 108)
W20260810 22:58:47.370543 15686 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:47.370390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:44643 (local address 127.0.0.1:49348)
0810 22:58:47.370453 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.370502 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:44643: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 22:58:47.370685 14749 catalog_manager.cc:4584] TS 506082469df84a7780a3064d7d5f7b03 (127.6.47.129:44643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 365028b3e3ea4b0dbcf5e18c81a3edc7: Network error: Client connection negotiation failed: client connection to 127.6.47.129:44643: connect: Connection reset by peer (error 104)
I20260810 22:58:47.370847 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 15154
I20260810 22:58:47.423341 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.423181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:44643 (local address 127.0.0.1:49352)
0810 22:58:47.423261 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.423297 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:44643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 22:58:47.423766 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.423661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:44643 (local address 127.0.0.1:49368)
0810 22:58:47.423712 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.423744 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:44643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 22:58:47.429399 15752 ts_tablet_manager.cc:1918] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:47.429515 15752 log.cc:1196] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/365028b3e3ea4b0dbcf5e18c81a3edc7
I20260810 22:58:47.429812 15752 ts_tablet_manager.cc:1939] T 365028b3e3ea4b0dbcf5e18c81a3edc7 P 2f7df9e28e3742afaeb062a20155c381: Deleting consensus metadata
I20260810 22:58:47.433091 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 15423
I20260810 22:58:47.433121 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.433002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:41171 (local address 127.0.0.1:54698)
0810 22:58:47.433059 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.433091 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.130:41171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 22:58:47.454669 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14717
I20260810 22:58:47.454711 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.454541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:39569 (local address 127.0.0.1:50064)
0810 22:58:47.454618 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.454660 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.131:39569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
2026-08-10T22:58:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (862 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 22:58:47.478402 6334 mini_oidc.cc:152] Starting JWKS server
I20260810 22:58:47.479561 6334 webserver.cc:460] Webserver started at https://127.0.0.1:32837/ using document root <none> and password file <none>
I20260810 22:58:47.479648 6334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:58:47.479789 6334 webserver.cc:460] Webserver started at http://127.0.0.1:44369/ using document root <none> and password file <none>
2026-08-10T22:58:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:47Z Disabled control of system clock
I20260810 22:58:47.489004 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:43465
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:41513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:43465
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32837/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/testchainca.cert with env {}
W20260810 22:58:47.576269 15761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:47.576486 15761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:47.576568 15761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:47.577766 15761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:47.577809 15761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:47.577822 15761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:47.577836 15761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:47.577847 15761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:58:47.577859 15761 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:58:47.577870 15761 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32837/jwks.json
I20260810 22:58:47.579538 15761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:43465
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:43465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32837/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:47.579751 15761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:47.579937 15761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:47.584976 15766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.585048 15769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.584976 15773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:47.585244 15761 server_base.cc:1034] running on GCE node
I20260810 22:58:47.585503 15761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:47.585754 15761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:47.586918 15761 hybrid_clock.cc:648] HybridClock initialized: now 1786402727586893 us; error 44 us; skew 500 ppm
I20260810 22:58:47.588171 15761 webserver.cc:460] Webserver started at http://127.6.47.190:42301/ using document root <none> and password file <none>
I20260810 22:58:47.588415 15761 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:47.588483 15761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:47.588636 15761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:47.589731 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "7889f2083b5e423b9d6aaacb6b05e8d6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "8a898430f303da6db9a60e07e4fce196"
server_key_iv: "e7efaeb7a9b28726ab77648c4d29c204"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.590113 15761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "7889f2083b5e423b9d6aaacb6b05e8d6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "8a898430f303da6db9a60e07e4fce196"
server_key_iv: "e7efaeb7a9b28726ab77648c4d29c204"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.591497 15761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:47.592244 15785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.592422 15761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:47.592546 15761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "7889f2083b5e423b9d6aaacb6b05e8d6"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "8a898430f303da6db9a60e07e4fce196"
server_key_iv: "e7efaeb7a9b28726ab77648c4d29c204"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.592621 15761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:47.606035 15761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:47.606331 15761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:47.606474 15761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:47.610569 15761 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:43465
I20260810 22:58:47.610607 15882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:43465 every 8 connection(s)
I20260810 22:58:47.610973 15761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:47.612017 15883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:47.613509 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 15761
I20260810 22:58:47.613619 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:47.613819 6334 external_mini_cluster.cc:1428] Setting key a0a3ae1ad929f047938c242dced6cbbc
I20260810 22:58:47.614993 15883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6: Bootstrap starting.
I20260810 22:58:47.615725 15883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:47.616052 15883 log.cc:824] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:47.616858 15883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6: No bootstrap required, opened a new log
I20260810 22:58:47.619122 15883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } }
I20260810 22:58:47.619323 15883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:47.619375 15883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7889f2083b5e423b9d6aaacb6b05e8d6, State: Initialized, Role: FOLLOWER
I20260810 22:58:47.619535 15883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } }
I20260810 22:58:47.619634 15883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:47.619689 15883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:47.619745 15883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:47.620186 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.614352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.6.47.190:43465)
0810 22:58:47.614552 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.614559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.614583 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:58:47.614983 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.614990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.615074 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.615205 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.615438 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.616403 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.617289 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.617904 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.618044 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.619390 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.619395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.619395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.619608 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.619644 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.619680 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.619747 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.619869 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.619978 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260810 22:58:47.621281 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:58:47.633499 15883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } }
I20260810 22:58:47.633687 15883 leader_election.cc:302] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7889f2083b5e423b9d6aaacb6b05e8d6; no voters:
I20260810 22:58:47.633893 15883 leader_election.cc:288] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:47.633976 15891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:47.634135 15891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [term 1 LEADER]: Becoming Leader. State: Replica: 7889f2083b5e423b9d6aaacb6b05e8d6, State: Running, Role: LEADER
I20260810 22:58:47.634253 15891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } }
I20260810 22:58:47.634258 15883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:47.634716 15895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7889f2083b5e423b9d6aaacb6b05e8d6. Latest consensus state: current_term: 1 leader_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } } }
I20260810 22:58:47.634812 15895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:47.634958 15894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7889f2083b5e423b9d6aaacb6b05e8d6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 43465 } } }
I20260810 22:58:47.635075 15900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:47.635644 15900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:47.635082 15894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:47.638502 15900 catalog_manager.cc:1347] Generated new cluster ID: 80d79096f66243b19d127c397e2dbc63
I20260810 22:58:47.638556 15900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:47.644865 15900 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:47.645500 15900 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:47.651262 15900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7889f2083b5e423b9d6aaacb6b05e8d6: Generated new TSK 0
I20260810 22:58:47.651450 15900 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 22:58:47.674400 15939 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 22:58:47.674613 15939 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 22:58:47.674796 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.659109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.6.47.190:43465)
0810 22:58:47.659187 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.659189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.659279 (+ 90us) server_negotiation.cc:407] Connection header received
0810 22:58:47.659386 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.659386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.659397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.659463 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:58:47.659658 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.660183 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.661366 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.661548 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.661624 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.661866 (+ 242us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:58:47.674504 (+ 12638us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 22:58:47.674711 (+ 207us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 22:58:47.674881 15888 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 22:58:47.675391 15938 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:47.658970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:43465 (local address 127.0.0.1:34248)
0810 22:58:47.659193 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.659205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.659279 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.659506 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.659507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.659510 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:58:47.659584 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:47.659587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.660284 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.660286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:47.661281 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:47.661284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.661645 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.661646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:47.661723 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.661746 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:58:47.674804 (+ 13058us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 22:58:47.674940 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.47.190:43465: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.47.190:43465: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32837/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 22:58:47.675848 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 15761
2026-08-10T22:58:47Z chronyd exiting
I20260810 22:58:47.725176 6334 test_util.cc:175] -----------------------------------------------
I20260810 22:58:47.725256 6334 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786402701710467-6334-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (248 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 22:58:47.726030 6334 mini_oidc.cc:152] Starting JWKS server
I20260810 22:58:47.726687 6334 webserver.cc:460] Webserver started at https://127.0.0.1:38789/ using document root <none> and password file <none>
I20260810 22:58:47.726727 6334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:58:47.726874 6334 webserver.cc:460] Webserver started at http://127.0.0.1:41527/ using document root <none> and password file <none>
2026-08-10T22:58:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:47Z Disabled control of system clock
I20260810 22:58:47.735785 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33377
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:46165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:33377
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38789/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/testchainca.cert with env {}
W20260810 22:58:47.829905 15955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:47.830214 15955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:47.830241 15955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:47.831633 15955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:47.831684 15955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:47.831699 15955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:47.831736 15955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:47.831772 15955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:58:47.831794 15955 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:58:47.831828 15955 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38789/jwks.json
I20260810 22:58:47.833770 15955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:33377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38789/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:47.834080 15955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:47.834342 15955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:47.839437 15970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.839470 15966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:47.839437 15967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:47.839705 15955 server_base.cc:1034] running on GCE node
I20260810 22:58:47.840018 15955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:47.840392 15955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:47.841560 15955 hybrid_clock.cc:648] HybridClock initialized: now 1786402727841516 us; error 58 us; skew 500 ppm
I20260810 22:58:47.843112 15955 webserver.cc:460] Webserver started at http://127.6.47.190:36841/ using document root <none> and password file <none>
I20260810 22:58:47.843417 15955 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:47.843487 15955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:47.843628 15955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:47.844816 15955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "ae675b3f743d489488e691c4185588f2"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "d634453bae3a842626c9e79b0be5a73a"
server_key_iv: "a090af81ee934b50b8ac8533c0774993"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.845171 15955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "ae675b3f743d489488e691c4185588f2"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "d634453bae3a842626c9e79b0be5a73a"
server_key_iv: "a090af81ee934b50b8ac8533c0774993"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.846408 15955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:47.847090 15984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:47.847259 15955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:47.847333 15955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "ae675b3f743d489488e691c4185588f2"
format_stamp: "Formatted at 2026-08-10 22:58:47 on dist-test-slave-rwrg"
server_key: "d634453bae3a842626c9e79b0be5a73a"
server_key_iv: "a090af81ee934b50b8ac8533c0774993"
server_key_version: "encryptionkey@0"
I20260810 22:58:47.847388 15955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:47.857728 15955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:47.857978 15955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:47.858079 15955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:47.862849 15955 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:33377
I20260810 22:58:47.862883 16086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:33377 every 8 connection(s)
I20260810 22:58:47.863255 15955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:47.863708 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 15955
I20260810 22:58:47.863824 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:47.863996 6334 external_mini_cluster.cc:1428] Setting key fc1e6f118410ae0c0ce3cdb121cf8d10
I20260810 22:58:47.864418 16089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:47.867764 16089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2: Bootstrap starting.
I20260810 22:58:47.868548 16089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:47.868902 16089 log.cc:824] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:47.869764 16089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2: No bootstrap required, opened a new log
I20260810 22:58:47.869865 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.864554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.6.47.190:33377)
0810 22:58:47.864770 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.864778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.864802 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:58:47.865234 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.865240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.865297 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.865438 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:47.865687 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.866546 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.867666 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.868116 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.868210 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.868988 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:47.869014 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:47.869015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:47.869266 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:47.869310 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:47.869326 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:47.869401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:47.869519 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 22:58:47.869624 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
I20260810 22:58:47.870849 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:58:47.871716 16089 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } }
I20260810 22:58:47.871889 16089 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:47.871939 16089 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae675b3f743d489488e691c4185588f2, State: Initialized, Role: FOLLOWER
I20260810 22:58:47.872071 16089 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } }
I20260810 22:58:47.872152 16089 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:47.872203 16089 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:47.872253 16089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:47.934540 16089 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } }
I20260810 22:58:47.934772 16089 leader_election.cc:302] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae675b3f743d489488e691c4185588f2; no voters:
I20260810 22:58:47.935043 16089 leader_election.cc:288] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:47.935145 16098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:47.935348 16098 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [term 1 LEADER]: Becoming Leader. State: Replica: ae675b3f743d489488e691c4185588f2, State: Running, Role: LEADER
I20260810 22:58:47.935362 16089 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:47.935459 16098 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } }
I20260810 22:58:47.935909 16099 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae675b3f743d489488e691c4185588f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } } }
I20260810 22:58:47.936026 16099 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:47.935918 16100 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae675b3f743d489488e691c4185588f2. Latest consensus state: current_term: 1 leader_uuid: "ae675b3f743d489488e691c4185588f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae675b3f743d489488e691c4185588f2" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33377 } } }
I20260810 22:58:47.936076 16100 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:47.936647 16110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:47.937283 16110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:47.939004 16110 catalog_manager.cc:1347] Generated new cluster ID: 32492ae4f8994e469abf45431b9d6b3c
I20260810 22:58:47.939077 16110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:47.949579 16110 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:47.950104 16110 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:47.956741 16110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae675b3f743d489488e691c4185588f2: Generated new TSK 0
I20260810 22:58:47.956931 16110 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 22:58:47.984575 16142 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 22:58:47.984668 16142 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 22:58:47.984998 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:47.970388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.6.47.190:33377)
0810 22:58:47.970437 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:47.970440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:47.970552 (+ 112us) server_negotiation.cc:407] Connection header received
0810 22:58:47.970593 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:47.970594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:47.970615 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:47.970703 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:58:47.970842 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.971671 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.972417 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.972766 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.972841 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.972996 (+ 155us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:58:47.984761 (+ 11765us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 22:58:47.984924 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 22:58:47.985059 16094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 22:58:47.985085 16141 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:47.970322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:33377 (local address 127.0.0.1:46116)
0810 22:58:47.970464 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:47.970482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:47.970540 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:47.970710 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:47.970711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:47.970713 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:58:47.970768 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:47.970770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.971752 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.971753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:47.972332 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:47.972333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:47.972845 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:47.972846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:47.972889 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:47.972893 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:58:47.984881 (+ 11988us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 22:58:47.984983 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.47.190:33377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38789/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260810 22:58:47.985430 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 15955
2026-08-10T22:58:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (305 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 22:58:48.031438 6334 mini_oidc.cc:152] Starting JWKS server
I20260810 22:58:48.032351 6334 webserver.cc:460] Webserver started at https://127.0.0.1:42025/ using document root <none> and password file <none>
I20260810 22:58:48.032395 6334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 22:58:48.032625 6334 webserver.cc:460] Webserver started at http://127.0.0.1:40719/ using document root <none> and password file <none>
2026-08-10T22:58:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:48Z Disabled control of system clock
I20260810 22:58:48.042584 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34933
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:34933
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42025/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 22:58:48.129383 16154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:48.129585 16154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:48.129614 16154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:48.130949 16154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:48.130997 16154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:48.131011 16154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:48.131052 16154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:48.131078 16154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:58:48.131100 16154 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:58:48.131129 16154 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42025/jwks.json
I20260810 22:58:48.132710 16154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:34933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42025/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:48.132990 16154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:48.133348 16154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:48.138646 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:48.138681 16165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:48.138653 16163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:48.138926 16154 server_base.cc:1034] running on GCE node
I20260810 22:58:48.139253 16154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:48.139570 16154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:48.140741 16154 hybrid_clock.cc:648] HybridClock initialized: now 1786402728140719 us; error 68 us; skew 500 ppm
I20260810 22:58:48.142066 16154 webserver.cc:460] Webserver started at http://127.6.47.190:39839/ using document root <none> and password file <none>
I20260810 22:58:48.142342 16154 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:48.142419 16154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:48.142575 16154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:48.143682 16154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "e57ab31bc0bf4b90b61d420076603c42"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "8672f0e15e0fe42529e4e6b37cce6551"
server_key_iv: "05c57fb8ff3be21c4aea6a622b173f39"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.144055 16154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "e57ab31bc0bf4b90b61d420076603c42"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "8672f0e15e0fe42529e4e6b37cce6551"
server_key_iv: "05c57fb8ff3be21c4aea6a622b173f39"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.145567 16154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:48.146394 16177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:48.146569 16154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:48.146684 16154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "e57ab31bc0bf4b90b61d420076603c42"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "8672f0e15e0fe42529e4e6b37cce6551"
server_key_iv: "05c57fb8ff3be21c4aea6a622b173f39"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.146777 16154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:48.169826 16154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:48.170179 16154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:48.170343 16154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:48.174844 16154 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:34933
I20260810 22:58:48.174882 16269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:34933 every 8 connection(s)
I20260810 22:58:48.175251 16154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:48.176126 16272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:48.178086 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 16154
I20260810 22:58:48.178203 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:48.178395 6334 external_mini_cluster.cc:1428] Setting key ac58dacb7425ce0f03cecc9956e44f7b
I20260810 22:58:48.179491 16272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42: Bootstrap starting.
I20260810 22:58:48.180256 16272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:48.180634 16272 log.cc:824] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:48.181401 16272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42: No bootstrap required, opened a new log
I20260810 22:58:48.183733 16272 raft_consensus.cc:348] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } }
I20260810 22:58:48.183902 16272 raft_consensus.cc:374] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:48.183943 16272 raft_consensus.cc:729] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e57ab31bc0bf4b90b61d420076603c42, State: Initialized, Role: FOLLOWER
I20260810 22:58:48.184122 16272 consensus_queue.cc:260] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } }
I20260810 22:58:48.184206 16272 raft_consensus.cc:388] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:48.184280 16272 raft_consensus.cc:482] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:48.184338 16272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:48.184535 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:48.178951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.6.47.190:34933)
0810 22:58:48.179178 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:48.179186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:48.179212 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:58:48.179686 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:48.179693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:48.179756 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:48.179876 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:48.180154 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.181253 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.182358 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.182783 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.182929 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.183617 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:48.183620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:48.183621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:48.183789 (+ 168us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:48.183822 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:48.183833 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:48.183885 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:48.184029 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 22:58:48.184123 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260810 22:58:48.185518 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:58:48.236047 16272 raft_consensus.cc:504] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } }
I20260810 22:58:48.236240 16272 leader_election.cc:302] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e57ab31bc0bf4b90b61d420076603c42; no voters:
I20260810 22:58:48.236562 16272 leader_election.cc:288] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:48.236593 16280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:48.236764 16280 raft_consensus.cc:686] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [term 1 LEADER]: Becoming Leader. State: Replica: e57ab31bc0bf4b90b61d420076603c42, State: Running, Role: LEADER
I20260810 22:58:48.236886 16280 consensus_queue.cc:237] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } }
I20260810 22:58:48.236936 16272 sys_catalog.cc:564] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:48.237263 16282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e57ab31bc0bf4b90b61d420076603c42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } } }
I20260810 22:58:48.237362 16282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:48.237859 16290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:48.238283 16283 sys_catalog.cc:455] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e57ab31bc0bf4b90b61d420076603c42. Latest consensus state: current_term: 1 leader_uuid: "e57ab31bc0bf4b90b61d420076603c42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e57ab31bc0bf4b90b61d420076603c42" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34933 } } }
I20260810 22:58:48.238397 16290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:48.238543 16283 sys_catalog.cc:458] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:48.242147 16290 catalog_manager.cc:1347] Generated new cluster ID: 684d3c4ce58b44cf8b465e90b5a0424a
I20260810 22:58:48.242214 16290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:48.258569 16290 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:48.259255 16290 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:48.266930 16290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e57ab31bc0bf4b90b61d420076603c42: Generated new TSK 0
I20260810 22:58:48.267139 16290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:58:48.281569 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:48.277735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.6.47.190:34933)
0810 22:58:48.277783 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:48.277785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:48.277878 (+ 93us) server_negotiation.cc:407] Connection header received
0810 22:58:48.277940 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:48.277941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:48.277951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:48.278009 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:58:48.278160 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.278696 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.279874 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.280061 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.280111 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.280327 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:58:48.281356 (+ 1029us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42025/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 22:58:48.281487 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44860: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42025/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260810 22:58:48.281638 16277 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44860: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42025/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 22:58:48.281880 16336 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:48.277601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:34933 (local address 127.0.0.1:44860)
0810 22:58:48.277794 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:48.277805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:48.277878 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:48.278028 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:48.278029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:48.278031 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:58:48.278090 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:48.278092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.278787 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.278788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:48.279783 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:48.279786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.280142 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.280143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:48.280207 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.280214 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:58:48.281472 (+ 1258us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42025/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 22:58:48.281588 (+ 116us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.47.190:34933: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42025/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":127,"threads_started":1}
I20260810 22:58:48.282115 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 16154
E20260810 22:58:48.301431 16337 webserver.cc:533] Webserver: sslize failed:
E20260810 22:58:48.301535 16337 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T22:58:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (299 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T22:58:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:48Z Disabled control of system clock
I20260810 22:58:48.340374 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46347
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:35495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:46347
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/testchainca.cert with env {}
W20260810 22:58:48.429407 16342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:48.429658 16342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:48.429683 16342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:48.430910 16342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:48.430959 16342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:48.430979 16342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:48.431020 16342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:48.431053 16342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 22:58:48.431073 16342 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 22:58:48.431106 16342 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 22:58:48.432891 16342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:46347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:48.433319 16342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:48.433820 16342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:48.438750 16357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:48.438807 16350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:48.438756 16362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:48.438961 16342 server_base.cc:1034] running on GCE node
I20260810 22:58:48.439229 16342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:48.439555 16342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:48.440704 16342 hybrid_clock.cc:648] HybridClock initialized: now 1786402728440685 us; error 39 us; skew 500 ppm
I20260810 22:58:48.442323 16342 webserver.cc:460] Webserver started at http://127.6.47.190:37889/ using document root <none> and password file <none>
I20260810 22:58:48.442592 16342 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:48.442642 16342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:48.442812 16342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:48.443921 16342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "f73ea1e65bb149b09904612e55458f92"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "1912b6b369e6cfde81a4bf169bf90d8a"
server_key_iv: "ae3f73ce9a28843ba815710af177834b"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.444305 16342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "f73ea1e65bb149b09904612e55458f92"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "1912b6b369e6cfde81a4bf169bf90d8a"
server_key_iv: "ae3f73ce9a28843ba815710af177834b"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.445683 16342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:48.446382 16373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:48.446554 16342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:48.446686 16342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "f73ea1e65bb149b09904612e55458f92"
format_stamp: "Formatted at 2026-08-10 22:58:48 on dist-test-slave-rwrg"
server_key: "1912b6b369e6cfde81a4bf169bf90d8a"
server_key_iv: "ae3f73ce9a28843ba815710af177834b"
server_key_version: "encryptionkey@0"
I20260810 22:58:48.446759 16342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:48.460244 16342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:48.460712 16342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:48.460880 16342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:48.465296 16342 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:46347
I20260810 22:58:48.465349 16475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:46347 every 8 connection(s)
I20260810 22:58:48.465747 16342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:48.466691 16478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:48.469048 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92: Bootstrap starting.
I20260810 22:58:48.469678 16478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:48.469988 16478 log.cc:824] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:48.470770 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92: No bootstrap required, opened a new log
I20260810 22:58:48.472368 16478 raft_consensus.cc:348] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } }
I20260810 22:58:48.472554 16478 raft_consensus.cc:374] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:48.472610 16478 raft_consensus.cc:729] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f73ea1e65bb149b09904612e55458f92, State: Initialized, Role: FOLLOWER
I20260810 22:58:48.472792 16478 consensus_queue.cc:260] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } }
I20260810 22:58:48.472904 16478 raft_consensus.cc:388] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:48.472945 16478 raft_consensus.cc:482] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:48.472997 16478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:48.473537 16478 raft_consensus.cc:504] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } }
I20260810 22:58:48.473663 16478 leader_election.cc:302] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f73ea1e65bb149b09904612e55458f92; no voters:
I20260810 22:58:48.473851 16478 leader_election.cc:288] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:48.473898 16484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:48.474116 16478 sys_catalog.cc:564] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:48.474123 16484 raft_consensus.cc:686] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [term 1 LEADER]: Becoming Leader. State: Replica: f73ea1e65bb149b09904612e55458f92, State: Running, Role: LEADER
I20260810 22:58:48.474249 16484 consensus_queue.cc:237] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } }
I20260810 22:58:48.474655 16486 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f73ea1e65bb149b09904612e55458f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } } }
I20260810 22:58:48.474795 16488 sys_catalog.cc:455] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f73ea1e65bb149b09904612e55458f92. Latest consensus state: current_term: 1 leader_uuid: "f73ea1e65bb149b09904612e55458f92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f73ea1e65bb149b09904612e55458f92" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46347 } } }
I20260810 22:58:48.474875 16488 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:48.474836 16486 sys_catalog.cc:458] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:48.475117 16507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:48.475392 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 16342
I20260810 22:58:48.475500 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:48.475689 6334 external_mini_cluster.cc:1428] Setting key 33389c9943cce5f4ab8e953cb1d327a0
I20260810 22:58:48.476305 16507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:48.478193 16507 catalog_manager.cc:1347] Generated new cluster ID: cd359dc6372c4da3b2717a5f05a45881
I20260810 22:58:48.478250 16507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:48.481684 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:48.476223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45440 (local address 127.6.47.190:46347)
0810 22:58:48.476510 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:48.476521 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:48.477220 (+ 699us) server_negotiation.cc:407] Connection header received
0810 22:58:48.477339 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:48.477347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:48.477420 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:48.477584 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:48.477837 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.478844 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.479650 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.480086 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.480232 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.481001 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:48.481005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:48.481005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:58:48.481196 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:48.481228 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:58:48.481238 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:48.481311 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:48.481382 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 22:58:48.481461 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260810 22:58:48.482623 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 22:58:48.496110 16507 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:48.496742 16507 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:48.501539 16507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f73ea1e65bb149b09904612e55458f92: Generated new TSK 0
I20260810 22:58:48.501755 16507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:58:48.951076 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:48.511846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.6.47.190:46347)
0810 22:58:48.511906 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:48.511908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:48.512026 (+ 118us) server_negotiation.cc:407] Connection header received
0810 22:58:48.512072 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:48.512073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:48.512086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:48.512180 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0810 22:58:48.512317 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.513182 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.513952 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.514287 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.514360 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.514514 (+ 154us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 22:58:48.950825 (+436311us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 22:58:48.951015 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45454: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 22:58:48.951141 16525 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45454: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 22:58:48.951263 16542 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:48.511735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:46347 (local address 127.0.0.1:45454)
0810 22:58:48.511934 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:48.511946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:48.512021 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:48.512193 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:48.512194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:48.512196 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0810 22:58:48.512248 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:48.512250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.513273 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.513273 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:48.513867 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:48.513868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:48.514372 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:48.514373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:48.514414 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:48.514419 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 22:58:48.950994 (+436575us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 22:58:48.951121 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.47.190:46347: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":126,"threads_started":1}
I20260810 22:58:48.951700 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 16342
2026-08-10T22:58:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (645 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:49 dist-test-slave-rwrg krb5kdc[16568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:49 dist-test-slave-rwrg krb5kdc[16568](info): set up 2 sockets
Aug 10 22:58:49 dist-test-slave-rwrg krb5kdc[16568](info): commencing operation
krb5kdc: starting...
W20260810 22:58:50.475553 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:50 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:58:50.603024 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39683
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:50.684120 16595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:50.684326 16595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:50.684348 16595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:50.685516 16595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:50.685559 16595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:50.685575 16595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:50.685586 16595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:50.685598 16595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:50.687000 16595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:39683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:50.687222 16595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:50.687405 16595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:50.692291 16608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:50.692386 16595 server_base.cc:1034] running on GCE node
W20260810 22:58:50.692260 16611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:50.692281 16605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:50.692799 16595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:50.693044 16595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:50.694190 16595 hybrid_clock.cc:648] HybridClock initialized: now 1786402730694164 us; error 37 us; skew 500 ppm
Aug 10 22:58:50 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402730, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:50.699561 16595 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:58:50.700090 16595 webserver.cc:460] Webserver started at http://127.6.47.190:45829/ using document root <none> and password file <none>
I20260810 22:58:50.700330 16595 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:50.700377 16595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:50.700557 16595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:50.701632 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "6485f7fa530448adabf6acd8ca75765b"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "f800d0039f5bfb39e9da2e06602b7d72"
server_key_iv: "88c188d34ebedf15aad5ed2923337e9b"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.702024 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "6485f7fa530448adabf6acd8ca75765b"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "f800d0039f5bfb39e9da2e06602b7d72"
server_key_iv: "88c188d34ebedf15aad5ed2923337e9b"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.703377 16595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:58:50.704108 16623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:50.704286 16595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:50.704399 16595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "6485f7fa530448adabf6acd8ca75765b"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "f800d0039f5bfb39e9da2e06602b7d72"
server_key_iv: "88c188d34ebedf15aad5ed2923337e9b"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.704476 16595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:50.713342 16595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:50.715005 16595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:50.715163 16595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:50.719126 16595 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:39683
I20260810 22:58:50.719161 16727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:39683 every 8 connection(s)
I20260810 22:58:50.719525 16595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:50.720405 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:50.722694 16728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b: Bootstrap starting.
I20260810 22:58:50.723268 16728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:50.723553 16728 log.cc:824] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:50.724229 16728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b: No bootstrap required, opened a new log
I20260810 22:58:50.725798 16728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } }
I20260810 22:58:50.725924 16728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:50.725943 16728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6485f7fa530448adabf6acd8ca75765b, State: Initialized, Role: FOLLOWER
I20260810 22:58:50.726091 16728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } }
I20260810 22:58:50.726146 16728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:50.726207 16728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:50.726256 16728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:50.726783 16728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } }
I20260810 22:58:50.726897 16728 leader_election.cc:302] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6485f7fa530448adabf6acd8ca75765b; no voters:
I20260810 22:58:50.727056 16728 leader_election.cc:288] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:50.727149 16733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:50.727285 16728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:50.727537 16733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [term 1 LEADER]: Becoming Leader. State: Replica: 6485f7fa530448adabf6acd8ca75765b, State: Running, Role: LEADER
I20260810 22:58:50.727656 16733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } }
I20260810 22:58:50.728080 16736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6485f7fa530448adabf6acd8ca75765b. Latest consensus state: current_term: 1 leader_uuid: "6485f7fa530448adabf6acd8ca75765b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } } }
I20260810 22:58:50.728190 16736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:50.728406 16745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:50.728534 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 16595
I20260810 22:58:50.728642 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:50.728873 6334 external_mini_cluster.cc:1428] Setting key d22afa29b571d113c3f0042c4a015758
I20260810 22:58:50.729013 16734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6485f7fa530448adabf6acd8ca75765b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6485f7fa530448adabf6acd8ca75765b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39683 } } }
I20260810 22:58:50.729105 16734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:50.729233 16745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:50.730849 16745 catalog_manager.cc:1347] Generated new cluster ID: da5b5267c4504bda9df71757aeffba36
I20260810 22:58:50.730902 16745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:50.737838 16745 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:50.738564 16745 catalog_manager.cc:1504] Loading token signing keys...
Aug 10 22:58:50 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:50.743768 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:50.729400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.6.47.190:39683)
0810 22:58:50.729617 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:50.729626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:50.732833 (+ 3207us) server_negotiation.cc:407] Connection header received
0810 22:58:50.732959 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:50.732969 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:50.733035 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:50.733206 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:50.733934 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.735457 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.736459 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.737111 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.737234 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:50.740106 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:50.740110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:50.740111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:50.740123 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:50.742144 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:50.742552 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:50.742555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:50.742555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:50.742664 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:50.742877 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:50.742878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:50.742878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:50.743135 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:50.743276 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:50.743381 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 22:58:50.743516 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260810 22:58:50.744361 16745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6485f7fa530448adabf6acd8ca75765b: Generated new TSK 0
I20260810 22:58:50.744542 16745 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:58:50.775003 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:50.858625 16772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:50.858865 16772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:50.858906 16772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:50.860163 16772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:50.860230 16772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:50.860275 16772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:50.861877 16772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:50.862156 16772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:50.862401 16772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:50.867453 16785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:50.867453 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:50.867612 16784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:50.868163 16772 server_base.cc:1034] running on GCE node
I20260810 22:58:50.868346 16772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:50.868623 16772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:50.869781 16772 hybrid_clock.cc:648] HybridClock initialized: now 1786402730869762 us; error 34 us; skew 500 ppm
Aug 10 22:58:50 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402730, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:50.872251 16772 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:58:50.872735 16772 webserver.cc:460] Webserver started at http://127.6.47.129:45935/ using document root <none> and password file <none>
I20260810 22:58:50.872978 16772 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:50.873025 16772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:50.873204 16772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:50.874302 16772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "a9c493cb608f4730bba26168b654d940"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "cc49375aac940622b56b88bc5ebb79fe"
server_key_iv: "782b1a01b68d6ba26674694fd3146ad0"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.874660 16772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "a9c493cb608f4730bba26168b654d940"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "cc49375aac940622b56b88bc5ebb79fe"
server_key_iv: "782b1a01b68d6ba26674694fd3146ad0"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.875998 16772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:50.876807 16804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:50.877002 16772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:50.877099 16772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "a9c493cb608f4730bba26168b654d940"
format_stamp: "Formatted at 2026-08-10 22:58:50 on dist-test-slave-rwrg"
server_key: "cc49375aac940622b56b88bc5ebb79fe"
server_key_iv: "782b1a01b68d6ba26674694fd3146ad0"
server_key_version: "encryptionkey@0"
I20260810 22:58:50.877183 16772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:50.892800 16772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:50.894574 16772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:50.894744 16772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:50.894981 16772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:50.895318 16772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:50.895375 16772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:50.895409 16772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:50.895437 16772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:50.901597 16772 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40491
I20260810 22:58:50.901633 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40491 every 8 connection(s)
I20260810 22:58:50.901988 16772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 22:58:50 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402730, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:50.910497 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 16772
I20260810 22:58:50.910624 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:50.910852 6334 external_mini_cluster.cc:1428] Setting key e6631d7086be2c089f41a296749153d4
I20260810 22:58:50.911327 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:50.902728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:53489 (local address 127.6.47.190:39683)
0810 22:58:50.902774 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:50.902776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:50.903003 (+ 227us) server_negotiation.cc:407] Connection header received
0810 22:58:50.903902 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:50.903903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:50.903915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:50.903974 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:50.904637 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.905119 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.906041 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.906236 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.906282 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:50.908597 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:50.908599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:50.908599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:50.908606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:50.909840 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:50.910198 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:50.910199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:50.910199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:50.910252 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:50.910557 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:50.910558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:50.910558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:50.910745 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:50.910798 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:50.911149 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 22:58:50.911234 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:58:50.911500 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:50.902693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39683 (local address 127.6.47.129:53489)
0810 22:58:50.902866 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:50.902907 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:50.903818 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:50.904052 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:50.904055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:50.904304 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:50.904552 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:50.904558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.905197 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.905197 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:50.905969 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:50.905971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:50.906305 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:50.906306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:50.906403 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:50.906919 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:50.906927 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:50.908455 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:50.909914 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:50.909921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:50.909923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:50.910132 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:50.910325 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:50.910326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:50.910327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:50.910487 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:50.910799 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:50.910806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:50.910971 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 22:58:50.911097 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 22:58:50.911284 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260810 22:58:50.912261 17008 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39683
I20260810 22:58:50.912372 17008 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:50.912607 17008 heartbeater.cc:511] Master 127.6.47.190:39683 requested a full tablet report, sending...
I20260810 22:58:50.913208 16654 ts_manager.cc:194] Registered new tserver with Master: a9c493cb608f4730bba26168b654d940 (127.6.47.129:40491)
I20260810 22:58:50.914115 16654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:53489
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:58:50.937332 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:51.018381 17020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:51.018616 17020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:51.018642 17020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:51.019806 17020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:51.019848 17020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:51.019874 17020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:51.021405 17020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:51.021669 17020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:51.021920 17020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:51.027058 17027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:51.027114 17032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:51.027065 17028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:51.027290 17020 server_base.cc:1034] running on GCE node
I20260810 22:58:51.027597 17020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:51.027865 17020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:51.029008 17020 hybrid_clock.cc:648] HybridClock initialized: now 1786402731028987 us; error 43 us; skew 500 ppm
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.031723 17020 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:58:51.032196 17020 webserver.cc:460] Webserver started at http://127.6.47.130:36717/ using document root <none> and password file <none>
I20260810 22:58:51.032407 17020 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:51.032449 17020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:51.032603 17020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:51.033627 17020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "e6d23c28c4924dd580de58219c5218f6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "0f7a68f2a5711682909ba9f1bc96c909"
server_key_iv: "f6a0db3f49708f5bb5a5c75bb0789416"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.033941 17020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "e6d23c28c4924dd580de58219c5218f6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "0f7a68f2a5711682909ba9f1bc96c909"
server_key_iv: "f6a0db3f49708f5bb5a5c75bb0789416"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.035223 17020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:51.035925 17040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.036101 17020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:51.036199 17020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "e6d23c28c4924dd580de58219c5218f6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "0f7a68f2a5711682909ba9f1bc96c909"
server_key_iv: "f6a0db3f49708f5bb5a5c75bb0789416"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.036258 17020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:51.052207 17020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:51.054203 17020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:51.054322 17020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:51.054527 17020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:51.054860 17020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:51.054894 17020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.054917 17020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:51.054965 17020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.061000 17020 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:45469
I20260810 22:58:51.061105 17249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:45469 every 8 connection(s)
I20260810 22:58:51.061412 17020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:51.062137 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 17020
I20260810 22:58:51.062276 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:51.062481 6334 external_mini_cluster.cc:1428] Setting key 255042d88f5b3ca8bab183db96bce323
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:51.072489 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.062293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:60915 (local address 127.6.47.190:39683)
0810 22:58:51.062351 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.062353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.062600 (+ 247us) server_negotiation.cc:407] Connection header received
0810 22:58:51.063912 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.063913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.063926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.063987 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.064901 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.065605 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.066577 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.066927 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.067021 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.069368 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:51.069370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:51.069370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:51.069379 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:51.071100 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.071489 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.071490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.071490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.071554 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.071796 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.071798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.071798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.071951 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:51.072027 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:51.072292 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 22:58:51.072383 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 22:58:51.072576 17252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.062246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39683 (local address 127.6.47.130:60915)
0810 22:58:51.062460 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:51.062511 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:51.063803 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:51.064061 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:51.064065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:51.064510 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:51.064807 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:51.064814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.065703 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.065704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:51.066492 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:51.066495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.067039 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.067040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:51.067165 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.067718 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:51.067734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:51.069272 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:51.071190 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:51.071197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:51.071198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:51.071409 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:51.071629 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:51.071630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:51.071630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:51.071724 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:51.072021 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:51.072025 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:51.072153 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 22:58:51.072236 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:58:51.072352 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
I20260810 22:58:51.073132 17250 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39683
I20260810 22:58:51.073246 17250 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:51.073421 17250 heartbeater.cc:511] Master 127.6.47.190:39683 requested a full tablet report, sending...
I20260810 22:58:51.073835 16654 ts_manager.cc:194] Registered new tserver with Master: e6d23c28c4924dd580de58219c5218f6 (127.6.47.130:45469)
I20260810 22:58:51.074275 16654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:60915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:58:51.091055 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:51.173061 17255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:51.173246 17255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:51.173269 17255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:51.174361 17255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:51.174403 17255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:51.174458 17255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:51.176040 17255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:51.176283 17255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:51.176574 17255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:51.181615 17264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:51.181656 17261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:51.181658 17262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:51.181859 17255 server_base.cc:1034] running on GCE node
I20260810 22:58:51.182107 17255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:51.182344 17255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:51.183514 17255 hybrid_clock.cc:648] HybridClock initialized: now 1786402731183497 us; error 49 us; skew 500 ppm
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.186074 17255 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:58:51.186542 17255 webserver.cc:460] Webserver started at http://127.6.47.131:33313/ using document root <none> and password file <none>
I20260810 22:58:51.186815 17255 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:51.186885 17255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:51.187017 17255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:51.188015 17255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "a4bcbd5b557049b397ae0c6373ac72e6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "27655be235eebf0478a80e67d3cc18b0"
server_key_iv: "d37b5c512bef9f5c2c25eaafd9e8e646"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.188369 17255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "a4bcbd5b557049b397ae0c6373ac72e6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "27655be235eebf0478a80e67d3cc18b0"
server_key_iv: "d37b5c512bef9f5c2c25eaafd9e8e646"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.189746 17255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:51.190454 17274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.190631 17255 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.190729 17255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "a4bcbd5b557049b397ae0c6373ac72e6"
format_stamp: "Formatted at 2026-08-10 22:58:51 on dist-test-slave-rwrg"
server_key: "27655be235eebf0478a80e67d3cc18b0"
server_key_iv: "d37b5c512bef9f5c2c25eaafd9e8e646"
server_key_version: "encryptionkey@0"
I20260810 22:58:51.190828 17255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:51.234355 17255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:51.236260 17255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:51.236423 17255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:51.236708 17255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:51.237040 17255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:51.237097 17255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.237133 17255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:51.237159 17255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:51.243089 17255 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:37385
I20260810 22:58:51.243144 17491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:37385 every 8 connection(s)
I20260810 22:58:51.243443 17255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:51.246778 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 17255
I20260810 22:58:51.246879 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:51.247030 6334 external_mini_cluster.cc:1428] Setting key 0d4f71c81fc4952e5282244df9e6329a
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:58:51.253111 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.244122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:56047 (local address 127.6.47.190:39683)
0810 22:58:51.244183 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.244184 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.244444 (+ 260us) server_negotiation.cc:407] Connection header received
0810 22:58:51.245557 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.245558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.245572 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.245638 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.246237 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.246726 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.248065 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.248290 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.248348 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.250461 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:51.250462 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:51.250462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:51.250468 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:51.251860 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.252234 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.252236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.252236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.252294 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.252566 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.252567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.252567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.252680 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:51.252737 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:51.252967 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 22:58:51.253041 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:58:51.253245 17495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.244110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39683 (local address 127.6.47.131:56047)
0810 22:58:51.244291 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:51.244355 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:51.245437 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:51.245716 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:51.245719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:51.245967 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:51.246146 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:51.246150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.246826 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.246828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:51.247956 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:51.247960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.248367 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.248367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:51.248473 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.248984 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:51.248992 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:51.250367 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:51.251927 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:51.251933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:51.251934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:51.252142 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:51.252363 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:51.252364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:51.252364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:51.252458 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:51.252740 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:51.252745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:51.252856 (+ 111us) client_negotiation.cc:769] Sending connection context
0810 22:58:51.252931 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 22:58:51.253064 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260810 22:58:51.253674 17492 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39683
I20260810 22:58:51.253763 17492 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:51.253964 17492 heartbeater.cc:511] Master 127.6.47.190:39683 requested a full tablet report, sending...
I20260810 22:58:51.254339 16654 ts_manager.cc:194] Registered new tserver with Master: a4bcbd5b557049b397ae0c6373ac72e6 (127.6.47.131:37385)
I20260810 22:58:51.254698 16654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:56047
I20260810 22:58:51.258731 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/test-admin.keytab.
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.274827 6334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 22:58:51.276422 17503 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:51.275589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.6.47.129:40491)
0810 22:58:51.275748 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.275759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.275769 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:51.275794 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.275796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.275808 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.275871 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.276313 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39474: BlockingRecv error: recv got EOF from 127.0.0.1:39474 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260810 22:58:51.276614 6334 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc' of type FILE
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.277474 6334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.278332 6334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:51.286139 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.278845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39482 (local address 127.6.47.129:40491)
0810 22:58:51.278882 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.278884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.278944 (+ 60us) server_negotiation.cc:407] Connection header received
0810 22:58:51.278981 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.278982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.278991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.279036 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.279409 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.280117 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.280914 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.281227 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.281281 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.283280 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:51.283289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:51.283289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:51.283295 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:51.284910 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.285284 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.285288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.285288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.285350 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.285556 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.285556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.285557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.285691 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:51.285835 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:51.285912 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 22:58:51.286029 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 22:58:51.287850 17503 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:51.287415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39492 (local address 127.6.47.129:40491)
0810 22:58:51.287480 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.287482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.287502 (+ 20us) server_negotiation.cc:407] Connection header received
0810 22:58:51.287550 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.287551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.287559 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.287601 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.287817 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39492: BlockingRecv error: recv got EOF from 127.0.0.1:39492 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 22:58:51.287937 6334 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc' of type FILE
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.288918 6334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.289812 6334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:51.297513 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.290337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.6.47.129:40491)
0810 22:58:51.290395 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.290397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.290442 (+ 45us) server_negotiation.cc:407] Connection header received
0810 22:58:51.290515 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.290516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.290529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.290597 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.290966 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.291447 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.292135 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.292344 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.292424 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.294593 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:51.294594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:51.294594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:51.294600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:51.296211 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.296654 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.296670 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.296670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.296738 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.296989 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.296990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.296991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.297104 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:51.297209 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:51.297311 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 22:58:51.297390 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 22:58:51.299139 17503 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:51.298536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.6.47.129:40491)
0810 22:58:51.298575 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.298577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.298664 (+ 87us) server_negotiation.cc:407] Connection header received
0810 22:58:51.298731 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.298732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.298741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.298787 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.299079 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39512: BlockingRecv error: recv got EOF from 127.0.0.1:39512 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 22:58:51.299203 6334 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786402701710467-6334-0/krb5kdc/krb5cc' of type FILE
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.300402 6334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:51.301676 6334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[16568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 22:58:51.311033 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:51.302285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.6.47.129:40491)
0810 22:58:51.302332 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:51.302334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:51.302413 (+ 79us) server_negotiation.cc:407] Connection header received
0810 22:58:51.302482 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:51.302484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:51.302493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:51.302542 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:51.302944 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.303464 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.304742 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:51.304936 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:51.304991 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:51.307948 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:51.307950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:51.307950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:51.307958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:51.309749 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.310103 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.310104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.310105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.310161 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:51.310379 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:51.310380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:51.310381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:51.310535 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:51.310639 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:51.310874 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 22:58:51.310953 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:58:51.311797 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 16772
I20260810 22:58:51.333784 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 17020
I20260810 22:58:51.346616 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 17255
I20260810 22:58:51.367846 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 16595
2026-08-10T22:58:51Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2234 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[17525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[17525](info): set up 2 sockets
Aug 10 22:58:51 dist-test-slave-rwrg krb5kdc[17525](info): commencing operation
krb5kdc: starting...
W20260810 22:58:52.649408 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:52 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab.
Entry for principal oryx/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab.
I20260810 22:58:52.771291 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:36187
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab
--principal=oryx/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:36187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:52.853474 17551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:52.853705 17551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:52.853729 17551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:52.854907 17551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:52.854950 17551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:52.854964 17551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:52.855006 17551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:52.855036 17551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:52.856645 17551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:36187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab
--principal=oryx/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:36187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:52.856902 17551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:52.857137 17551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:52.862155 17563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:52.862215 17561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:52.862434 17559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:52.862758 17551 server_base.cc:1034] running on GCE node
I20260810 22:58:52.863040 17551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:52.863348 17551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:52.864507 17551 hybrid_clock.cc:648] HybridClock initialized: now 1786402732864469 us; error 41 us; skew 500 ppm
Aug 10 22:58:52 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:52.866979 17551 init.cc:377] Logged in from keytab as oryx/127.6.47.190@KRBTEST.COM (short username oryx)
I20260810 22:58:52.867410 17551 webserver.cc:460] Webserver started at http://127.6.47.190:36531/ using document root <none> and password file <none>
I20260810 22:58:52.867640 17551 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:52.867687 17551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:52.867861 17551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:52.869033 17551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "8b28120137c2490091fa71c46ecfc396"
format_stamp: "Formatted at 2026-08-10 22:58:52 on dist-test-slave-rwrg"
server_key: "430cc945d67b98940ec9949141704020"
server_key_iv: "b22094134fc131af03962a67693bce9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:52.869421 17551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "8b28120137c2490091fa71c46ecfc396"
format_stamp: "Formatted at 2026-08-10 22:58:52 on dist-test-slave-rwrg"
server_key: "430cc945d67b98940ec9949141704020"
server_key_iv: "b22094134fc131af03962a67693bce9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:52.870784 17551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:52.871500 17579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:52.871681 17551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:52.871778 17551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "8b28120137c2490091fa71c46ecfc396"
format_stamp: "Formatted at 2026-08-10 22:58:52 on dist-test-slave-rwrg"
server_key: "430cc945d67b98940ec9949141704020"
server_key_iv: "b22094134fc131af03962a67693bce9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:52.871852 17551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:52.890322 17551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:52.892829 17551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:52.893005 17551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:52.897193 17551 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:36187
I20260810 22:58:52.897239 17687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:36187 every 8 connection(s)
I20260810 22:58:52.897611 17551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:52.898623 17688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:52.900969 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Bootstrap starting.
I20260810 22:58:52.901576 17688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:52.901901 17688 log.cc:824] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:52.902652 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: No bootstrap required, opened a new log
I20260810 22:58:52.904243 17688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } }
I20260810 22:58:52.904400 17688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:52.904449 17688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b28120137c2490091fa71c46ecfc396, State: Initialized, Role: FOLLOWER
I20260810 22:58:52.904639 17688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } }
I20260810 22:58:52.904731 17688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:52.904779 17688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:52.904831 17688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:52.905640 17688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } }
I20260810 22:58:52.905759 17688 leader_election.cc:302] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b28120137c2490091fa71c46ecfc396; no voters:
I20260810 22:58:52.905956 17688 leader_election.cc:288] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:52.905997 17695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:52.906139 17695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 LEADER]: Becoming Leader. State: Replica: 8b28120137c2490091fa71c46ecfc396, State: Running, Role: LEADER
I20260810 22:58:52.906229 17688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:52.906483 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 17551
I20260810 22:58:52.906603 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:52.906777 6334 external_mini_cluster.cc:1428] Setting key 6926e36ffc51b2be24e3bebb6b5a6a0a
I20260810 22:58:52.906243 17695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } }
I20260810 22:58:52.907684 17704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } }
I20260810 22:58:52.907799 17704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:52.907945 17707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b28120137c2490091fa71c46ecfc396. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } }
I20260810 22:58:52.908128 17707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:52.908069 17744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:52.908862 17744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:52.910512 17744 catalog_manager.cc:1347] Generated new cluster ID: 4f2c2286fd894f7b89178f1c114c0b7c
I20260810 22:58:52.910564 17744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:58:52 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:52.915982 17744 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:52.916656 17744 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:52.917588 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:52.907367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.6.47.190:36187)
0810 22:58:52.907709 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:52.907719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:52.907761 (+ 42us) server_negotiation.cc:407] Connection header received
0810 22:58:52.907890 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:52.907898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:52.908048 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:52.908205 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:52.909023 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:52.910222 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:52.911015 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:52.911410 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:52.911523 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:52.913994 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:52.913998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:52.913998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:52.914010 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:52.915792 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:52.916203 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:52.916206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:52.916207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:52.916302 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:52.916486 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:52.916487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:52.916488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:52.916768 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:52.916894 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:52.917263 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 22:58:52.917390 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":84,"threads_started":1}
I20260810 22:58:52.919351 17744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Generated new TSK 0
I20260810 22:58:52.919519 17744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
I20260810 22:58:52.947024 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:53.029829 17754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:53.030082 17754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:53.030126 17754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:53.031296 17754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:53.031363 17754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:53.031422 17754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:53.033048 17754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:53.033341 17754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:53.033586 17754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.038846 17765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.038875 17769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.038872 17767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.038892 17754 server_base.cc:1034] running on GCE node
I20260810 22:58:53.039197 17754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.039408 17754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.040573 17754 hybrid_clock.cc:648] HybridClock initialized: now 1786402733040554 us; error 40 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.043105 17754 init.cc:377] Logged in from keytab as oryx/127.6.47.129@KRBTEST.COM (short username oryx)
I20260810 22:58:53.043610 17754 webserver.cc:460] Webserver started at http://127.6.47.129:45741/ using document root <none> and password file <none>
I20260810 22:58:53.043887 17754 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.043957 17754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.044122 17754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:53.045308 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "c5a8217a27684e27aea8fd642e533813"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "e8538b6aa0d368bbf3d9424f649a7051"
server_key_iv: "7b9a918d2c972fc039f3573058ef8be4"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.045687 17754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "c5a8217a27684e27aea8fd642e533813"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "e8538b6aa0d368bbf3d9424f649a7051"
server_key_iv: "7b9a918d2c972fc039f3573058ef8be4"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.047043 17754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 22:58:53.047827 17788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.048005 17754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.048111 17754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "c5a8217a27684e27aea8fd642e533813"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "e8538b6aa0d368bbf3d9424f649a7051"
server_key_iv: "7b9a918d2c972fc039f3573058ef8be4"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.048209 17754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.063606 17754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.065634 17754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:53.065804 17754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.066076 17754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:53.066425 17754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:53.066502 17754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.066548 17754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:53.066574 17754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.072798 17754 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:34731
I20260810 22:58:53.072845 17982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:34731 every 8 connection(s)
I20260810 22:58:53.073192 17754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:53.082381 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.073966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:38877 (local address 127.6.47.190:36187)
0810 22:58:53.074012 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.074014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.074262 (+ 248us) server_negotiation.cc:407] Connection header received
0810 22:58:53.074867 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.074868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.074882 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.074931 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.076047 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.076618 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.077599 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.077815 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.077903 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.079928 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.079930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.079930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.079937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.081202 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.081564 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.081565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.081565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.081616 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.081828 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.081828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.081829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.081930 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.081997 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.082243 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.082314 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:58:53.082568 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.073917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:38877)
0810 22:58:53.074131 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.074178 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.074752 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.075002 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.075006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.075752 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.075946 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.075951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.076720 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.076721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.077504 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.077506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.077917 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.077918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.078045 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.078491 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.078500 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.079842 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.081274 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.081281 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.081283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.081498 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.081674 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.081674 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.081675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.081774 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.082003 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.082008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.082130 (+ 122us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.082214 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.082329 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260810 22:58:53.082685 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 17754
I20260810 22:58:53.082777 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:53.083014 6334 external_mini_cluster.cc:1428] Setting key c279a1408af94291d9f368654eb05a7b
I20260810 22:58:53.083168 17987 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:53.083284 17987 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:53.083473 17987 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:53.084064 17612 ts_manager.cc:194] Registered new tserver with Master: c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731)
I20260810 22:58:53.084739 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:38877
WARNING: no policy specified for oryx/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
I20260810 22:58:53.110715 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:53.194422 18001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:53.194643 18001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:53.194680 18001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:53.195775 18001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:53.195842 18001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:53.195887 18001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:53.197439 18001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:53.197696 18001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:53.197919 18001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.202780 18013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.202791 18007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.203084 18011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.203956 18001 server_base.cc:1034] running on GCE node
I20260810 22:58:53.204136 18001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.204325 18001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.205461 18001 hybrid_clock.cc:648] HybridClock initialized: now 1786402733205440 us; error 38 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.207969 18001 init.cc:377] Logged in from keytab as oryx/127.6.47.130@KRBTEST.COM (short username oryx)
I20260810 22:58:53.208389 18001 webserver.cc:460] Webserver started at http://127.6.47.130:41579/ using document root <none> and password file <none>
I20260810 22:58:53.208670 18001 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.208740 18001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.208875 18001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:53.209905 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "39fcf34c07912e35815d47dadb1309a7"
server_key_iv: "ec8bec2db6069875b98524b26cd6f63a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.210283 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "39fcf34c07912e35815d47dadb1309a7"
server_key_iv: "ec8bec2db6069875b98524b26cd6f63a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.211573 18001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:53.212314 18034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.212482 18001 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:53.212627 18001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "39fcf34c07912e35815d47dadb1309a7"
server_key_iv: "ec8bec2db6069875b98524b26cd6f63a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.212713 18001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.223371 18001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.225054 18001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:53.225212 18001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.225423 18001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:53.225708 18001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:53.225761 18001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.225798 18001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:53.225823 18001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.231323 18001 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:45555
I20260810 22:58:53.231405 18249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:45555 every 8 connection(s)
I20260810 22:58:53.231732 18001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:53.236224 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 18001
I20260810 22:58:53.236335 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:53.236573 6334 external_mini_cluster.cc:1428] Setting key 13d6d9662dbb041fab776df0f139238d
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:53.241690 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.232485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:55279 (local address 127.6.47.190:36187)
0810 22:58:53.232575 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.232578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.232752 (+ 174us) server_negotiation.cc:407] Connection header received
0810 22:58:53.233346 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.233347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.233360 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.233420 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.234391 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.234966 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.235967 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.236205 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.236283 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.238954 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.238955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.238955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.238961 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.240333 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.240803 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.240804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.240804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.240854 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.241115 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.241115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.241115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.241205 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.241249 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.241567 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.241632 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 22:58:53.242022 18254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.232429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:55279)
0810 22:58:53.232625 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.232671 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.233152 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.233493 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.233498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.234078 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.234277 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.234282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.235078 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.235079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.235875 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.235877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.236294 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.236295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.236460 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.237105 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.237113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.238868 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.240392 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.240399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.240401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.240725 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.240911 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.240912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.240912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.241049 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.241260 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.241266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.241416 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.241513 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.241738 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":114,"threads_started":1}
I20260810 22:58:53.242616 18251 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:53.242723 18251 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:53.242925 18251 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:53.243319 17612 ts_manager.cc:194] Registered new tserver with Master: f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:53.243692 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:55279
WARNING: no policy specified for oryx/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
I20260810 22:58:53.263083 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:53.345811 18259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:53.346115 18259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:53.346161 18259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:53.347306 18259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:53.347373 18259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:53.347416 18259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:53.348943 18259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:53.349211 18259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:53.349455 18259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.354701 18271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.354701 18268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.354799 18259 server_base.cc:1034] running on GCE node
W20260810 22:58:53.354774 18266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.355136 18259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.355336 18259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.356472 18259 hybrid_clock.cc:648] HybridClock initialized: now 1786402733356445 us; error 44 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.359329 18259 init.cc:377] Logged in from keytab as oryx/127.6.47.131@KRBTEST.COM (short username oryx)
I20260810 22:58:53.359874 18259 webserver.cc:460] Webserver started at http://127.6.47.131:44245/ using document root <none> and password file <none>
I20260810 22:58:53.360118 18259 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.360191 18259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.360348 18259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:53.361472 18259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "1c93871cf3e04ce2b9febf227378ed43"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3f991d0a64d372a664da7a7d8e547369"
server_key_iv: "1d15604f22b7b5e42f44c40a49627299"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.361850 18259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "1c93871cf3e04ce2b9febf227378ed43"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3f991d0a64d372a664da7a7d8e547369"
server_key_iv: "1d15604f22b7b5e42f44c40a49627299"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.363197 18259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:53.363884 18287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.364102 18259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:53.364209 18259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "1c93871cf3e04ce2b9febf227378ed43"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3f991d0a64d372a664da7a7d8e547369"
server_key_iv: "1d15604f22b7b5e42f44c40a49627299"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.364296 18259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.375871 18259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.377725 18259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:53.377878 18259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.378110 18259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:53.378410 18259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:53.378464 18259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.378506 18259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:53.378533 18259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.384346 18259 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:41759
I20260810 22:58:53.384393 18502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:41759 every 8 connection(s)
I20260810 22:58:53.384797 18259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:53.387854 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 18259
I20260810 22:58:53.387928 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:53.388131 6334 external_mini_cluster.cc:1428] Setting key 15b337204ef9588c4ef05057a47e5943
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:53.393833 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.385392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:49833 (local address 127.6.47.190:36187)
0810 22:58:53.385434 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.385435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.385636 (+ 201us) server_negotiation.cc:407] Connection header received
0810 22:58:53.386170 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.386172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.386184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.386231 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.387139 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.387631 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.389014 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.389322 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.389380 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.391443 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.391444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.391445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.391450 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.392721 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.393069 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.393069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.393070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.393120 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.393348 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.393349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.393349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.393443 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.393491 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.393711 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.393773 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:53.394024 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.385356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:49833)
0810 22:58:53.385494 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.385556 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.386070 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.386295 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.386299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.386857 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.387037 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.387042 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.387753 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.387755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.388927 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.388930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.389401 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.389402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.389508 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.390042 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.390050 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.391362 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.392791 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.392797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.392799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.393004 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.393200 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.393200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.393201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.393295 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.393495 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.393500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.393603 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.393682 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.393793 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260810 22:58:53.394454 18503 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:53.394556 18503 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:53.394747 18503 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:53.395105 17612 ts_manager.cc:194] Registered new tserver with Master: 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:53.395452 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:49833
I20260810 22:58:53.400025 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.189.keytab.
Entry for principal oryx/127.6.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.189.keytab.
I20260810 22:58:53.426961 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:38843
--webserver_interface=127.6.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.189.keytab
--principal=oryx/127.6.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:36187,127.6.47.189:38843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:53.510478 18510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:53.510704 18510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:53.510725 18510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:53.511780 18510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:53.511814 18510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:53.511828 18510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:53.511840 18510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:53.511852 18510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:53.513319 18510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:36187,127.6.47.189:38843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.189.keytab
--principal=oryx/127.6.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:38843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
W20260810 22:58:53.513429 18510 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.47.190:36187,127.6.47.189:38843'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 22:58:53.513540 18510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:53.513717 18510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.518757 18521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.518730 18516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.518738 18517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.519069 18510 server_base.cc:1034] running on GCE node
I20260810 22:58:53.519332 18510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.519634 18510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.520798 18510 hybrid_clock.cc:648] HybridClock initialized: now 1786402733520800 us; error 42 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.523327 18510 init.cc:377] Logged in from keytab as oryx/127.6.47.189@KRBTEST.COM (short username oryx)
I20260810 22:58:53.523764 18510 webserver.cc:460] Webserver started at http://127.6.47.189:34915/ using document root <none> and password file <none>
I20260810 22:58:53.524010 18510 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.524081 18510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.524251 18510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:53.525405 18510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/instance:
uuid: "cba40bc5ebbe479799858da66d24f307"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3b4f77e1899a4ec378001869e96b00c7"
server_key_iv: "c62671be6c57d34e22bdd464ba13b419"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.525789 18510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal/instance:
uuid: "cba40bc5ebbe479799858da66d24f307"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3b4f77e1899a4ec378001869e96b00c7"
server_key_iv: "c62671be6c57d34e22bdd464ba13b419"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.527170 18510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 22:58:53.527926 18537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.528115 18510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.528231 18510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "cba40bc5ebbe479799858da66d24f307"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3b4f77e1899a4ec378001869e96b00c7"
server_key_iv: "c62671be6c57d34e22bdd464ba13b419"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.528311 18510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.545840 18510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.547914 18510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:53.548098 18510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.552783 18510 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.189:38843
I20260810 22:58:53.552819 18636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.189:38843 every 8 connection(s)
I20260810 22:58:53.553184 18510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
I20260810 22:58:53.554339 18637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:53.555850 18637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.189@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:53.562386 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 18510
I20260810 22:58:53.565224 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.556536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.6.47.190:36187)
0810 22:58:53.556597 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.556599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.556849 (+ 250us) server_negotiation.cc:407] Connection header received
0810 22:58:53.557352 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.557354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.557367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.557420 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.558037 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.558552 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.559532 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.559734 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.559814 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.561985 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.561986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.561987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.562001 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.563638 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.564137 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.564138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.564138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.564196 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.564477 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.564478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.564478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.564669 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.564725 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.565038 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.565133 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 22:58:53.565322 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.556481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42290)
0810 22:58:53.556728 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.556764 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.557236 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.557492 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.557496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.557748 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.557943 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.557947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.558651 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.558653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.559440 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.559442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.559819 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.559821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.559942 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.560461 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.560470 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.561871 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.563732 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.563738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.563740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.564035 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.564277 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.564278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.564278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.564410 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.564737 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.564742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.564882 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.564999 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.565142 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":139,"threads_started":1}
I20260810 22:58:53.565929 18637 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.189@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
I20260810 22:58:53.572186 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.562993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.6.47.189:38843)
0810 22:58:53.563196 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.563202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.563228 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:58:53.563270 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.563273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.563289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.563368 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.563924 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.564930 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.565818 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.566216 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.566290 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.569052 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.569059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.569060 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.569065 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.570742 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.571176 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.571180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.571180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.571243 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.571444 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.571445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.571445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.571647 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.571863 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.571919 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.572050 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260810 22:58:53.574183 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.566179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41632)
0810 22:58:53.566230 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.566240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.566277 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.566476 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.566477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.566629 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.566690 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.566692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.567763 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.567764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.568282 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.568284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.568752 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.568753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.568829 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.569185 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.569189 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.570605 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.573163 (+ 2558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.573163 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.573164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.573328 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.573552 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.573553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.573553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.573610 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.573855 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.573856 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.573892 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.573960 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.574054 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 22:58:53.574330 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.566235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.6.47.189:38843)
0810 22:58:53.566374 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.566376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.566381 (+ 5us) server_negotiation.cc:407] Connection header received
0810 22:58:53.566397 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.566398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.566407 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.566470 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.566778 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.567681 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.568364 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.568665 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.568731 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.570686 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.570687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.570687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.570695 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.573051 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.573412 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.573413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.573413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.573476 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.573683 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.573684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.573684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.573788 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.573863 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.573989 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.574105 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260810 22:58:53.575685 18637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Bootstrap starting.
I20260810 22:58:53.576354 18637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:53.576658 18637 log.cc:824] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:53.577486 18637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: No bootstrap required, opened a new log
I20260810 22:58:53.578814 18637 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } }
I20260810 22:58:53.578933 18637 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:53.578955 18637 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba40bc5ebbe479799858da66d24f307, State: Initialized, Role: FOLLOWER
I20260810 22:58:53.579079 18637 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } }
I20260810 22:58:53.579332 18654 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } }
I20260810 22:58:53.579408 18637 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:53.579418 18654 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 22:58:53.581265 18671 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:58:53.581369 18671 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:58:53.583388 18510 master_runner.cc:428] Detected that this master cba40bc5ebbe479799858da66d24f307 is joining an existing cluster
I20260810 22:58:53.583483 18510 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.47.189:38843
I20260810 22:58:53.591066 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.584899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.6.47.190:36187)
0810 22:58:53.584970 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.584972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.584981 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:53.585023 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.585024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.585036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.585098 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.585440 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.585918 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.586710 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.586916 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.587000 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.588040 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.588041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.588042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.588049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.589779 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590196 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.590197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.590198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.590261 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590486 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.590487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.590487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.590630 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.590707 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.590894 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.590977 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 22:58:53.591104 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.584520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42292)
0810 22:58:53.584668 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.584681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.584765 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.585103 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.585104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.585290 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.585349 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.585350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.586046 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.586047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.586612 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.586614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.587016 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.587017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.587080 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.587555 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.587560 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.587868 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.589869 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.589871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.589871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.590114 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.590342 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.590343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.590411 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.590709 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.590710 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.590757 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.590838 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.590959 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260810 22:58:53.591841 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.584947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.6.47.189:38843)
0810 22:58:53.584981 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.584983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.584992 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:53.585067 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.585068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.585077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.585134 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.585538 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.586259 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.587309 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.587601 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.587652 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.588761 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.588762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.588763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.588769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.590507 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590880 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.590881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.590881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.590927 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.591091 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.591093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.591093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.591219 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.591266 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.591680 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.591763 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:53.592097 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.584775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41634)
0810 22:58:53.584906 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.584913 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.584998 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.585173 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.585174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.585410 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.585461 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.585463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.586346 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.586347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.587220 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.587223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.587720 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.587720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.587776 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.588250 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.588255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.588657 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.590569 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.590570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.590793 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.590988 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.590988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.590989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.591035 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.591533 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.591534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.591573 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.591915 (+ 342us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.591999 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.190@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
I20260810 22:58:53.600593 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.593578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41640)
0810 22:58:53.593702 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.593735 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.593819 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.593972 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.593975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.594321 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.594367 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.594369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.595025 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.595026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.595671 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.595674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.596050 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.596050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.596130 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.596594 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.596600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.597881 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.599363 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.599372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.599374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.599621 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.599832 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.599833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.599833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.599898 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.600179 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.600188 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.600240 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.600344 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.600465 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260810 22:58:53.600589 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.593745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.6.47.189:38843)
0810 22:58:53.593805 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.593807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.593815 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:53.593899 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.593900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.593910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.593965 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.594447 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.594937 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.595768 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.595973 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.596049 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.597971 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.597972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.597972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.597980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.599268 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.599700 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.599701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.599701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.599759 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.599961 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.599962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.599962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.600086 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.600168 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.600373 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.600457 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:58:53.601446 17612 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.47.189:38843: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cba40bc5ebbe479799858da66d24f307"
member_type: NON_VOTER
last_known_addr {
host: "127.6.47.189"
port: 38843
}
attrs {
promote: true
}
}
dest_uuid: "8b28120137c2490091fa71c46ecfc396"
cas_config_opid_index: -1
I20260810 22:58:53.601871 17612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } }
I20260810 22:58:53.602416 18686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cba40bc5ebbe479799858da66d24f307 (127.6.47.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.602715 18591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:53.603392 18686 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.47.189:38843
I20260810 22:58:53.603565 18686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.603665 18686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:53.603667 18591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 22:58:53.603735 18510 master.cc:537] Master@127.6.47.189:38843 shutting down...
I20260810 22:58:53.604240 18685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 22:58:53.607659 17581 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 22:58:53.607784 18510 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:53.607880 18510 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:53.607931 18510 tablet_replica.cc:330] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: stopping tablet replica
I20260810 22:58:53.615247 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.615111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41646)
0810 22:58:53.615167 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.615211 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:38843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 22:58:53.615375 17581 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 -> Peer cba40bc5ebbe479799858da66d24f307 (127.6.47.189:38843): Couldn't send request to peer cba40bc5ebbe479799858da66d24f307. Status: Network error: Client connection negotiation failed: client connection to 127.6.47.189:38843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:58:53.621153 18510 master.cc:552] Master@127.6.47.189:38843 shutdown complete.
I20260810 22:58:53.621206 18510 master_runner.cc:305] Clearing existing system tablet
I20260810 22:58:53.621342 18510 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.621392 18510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.622157 18510 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:53.622713 18699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.622887 18510 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.622952 18510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.622978 18510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "cba40bc5ebbe479799858da66d24f307"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3b4f77e1899a4ec378001869e96b00c7"
server_key_iv: "c62671be6c57d34e22bdd464ba13b419"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.623829 18510 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:53.625414 18510 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 22:58:53.625489 18510 log.cc:1196] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 22:58:53.625706 18510 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Deleting consensus metadata
I20260810 22:58:53.625828 18510 master_runner.cc:315] Copying system tablet from 127.6.47.190:36187
I20260810 22:58:53.626415 18510 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: tablet copy: Beginning tablet copy session from remote peer at address 127.6.47.190:36187
I20260810 22:58:53.632812 18715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.626753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42306)
0810 22:58:53.626891 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.626901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.626964 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.627103 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.627104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.627262 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.627314 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.627315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.628231 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.628231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.628834 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.628837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.629323 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.629324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.629372 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.629855 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.629859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.630147 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.631757 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.631758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.631758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.631915 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.632152 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.632153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.632153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.632211 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.632482 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.632483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.632575 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.632657 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.632739 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260810 22:58:53.632890 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.626825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.6.47.190:36187)
0810 22:58:53.626871 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.626873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.626952 (+ 79us) server_negotiation.cc:407] Connection header received
0810 22:58:53.627027 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.627028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.627038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.627093 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.627387 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.628145 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.628919 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.629214 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.629318 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.630254 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.630255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.630256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.630263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.631662 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.632001 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.632003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.632003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.632084 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.632274 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.632275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.632276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.632414 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.632518 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.632714 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.632799 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 22:58:53.633531 17656 tablet_copy_service.cc:140] P 8b28120137c2490091fa71c46ecfc396: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cba40bc5ebbe479799858da66d24f307 ({username='oryx', principal='oryx/127.6.47.189@KRBTEST.COM'} at 127.0.0.1:42306)
I20260810 22:58:53.633657 17656 tablet_copy_service.cc:161] P 8b28120137c2490091fa71c46ecfc396: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cba40bc5ebbe479799858da66d24f307 at {username='oryx', principal='oryx/127.6.47.189@KRBTEST.COM'} at 127.0.0.1:42306: session id = cba40bc5ebbe479799858da66d24f307-00000000000000000000000000000000
I20260810 22:58:53.634485 17656 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 22:58:53.635138 18510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 22:58:53.636484 18510 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: tablet copy: Starting download of 0 data blocks...
I20260810 22:58:53.636685 18510 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: tablet copy: Starting download of 1 WAL segments...
I20260810 22:58:53.637665 18510 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 22:58:53.638563 17656 tablet_copy_service.cc:342] P 8b28120137c2490091fa71c46ecfc396: Request end of tablet copy session cba40bc5ebbe479799858da66d24f307-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.47.189@KRBTEST.COM'} at 127.0.0.1:42306
I20260810 22:58:53.638629 17656 tablet_copy_service.cc:433] P 8b28120137c2490091fa71c46ecfc396: ending tablet copy session cba40bc5ebbe479799858da66d24f307-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cba40bc5ebbe479799858da66d24f307
I20260810 22:58:53.641225 18510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.642400 18718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.642519 18716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.642660 18510 server_base.cc:1034] running on GCE node
W20260810 22:58:53.644636 18720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.644801 18510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.644953 18510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.646091 18510 hybrid_clock.cc:648] HybridClock initialized: now 1786402733646091 us; error 33 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.647404 18510 init.cc:377] Logged in from keytab as oryx/127.6.47.189@KRBTEST.COM (short username oryx)
I20260810 22:58:53.647718 18510 webserver.cc:460] Webserver started at http://127.6.47.189:37271/ using document root <none> and password file <none>
I20260810 22:58:53.647850 18510 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.647899 18510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.648571 18510 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:53.649160 18729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.649318 18510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.649370 18510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "cba40bc5ebbe479799858da66d24f307"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3b4f77e1899a4ec378001869e96b00c7"
server_key_iv: "c62671be6c57d34e22bdd464ba13b419"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.649430 18510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.664259 18510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.665984 18510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.669417 18510 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.189:38843
I20260810 22:58:53.669706 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.189:38843 every 8 connection(s)
I20260810 22:58:53.669821 18510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
I20260810 22:58:53.670765 18824 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:58:53.671314 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Bootstrap starting.
I20260810 22:58:53.674542 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:58:53.674853 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: Bootstrap complete.
I20260810 22:58:53.675078 18824 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } }
I20260810 22:58:53.675170 18824 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cba40bc5ebbe479799858da66d24f307, State: Initialized, Role: LEARNER
I20260810 22:58:53.675225 18824 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } }
I20260810 22:58:53.675428 18824 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:53.676930 18828 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.677230 18828 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: LEARNER
I20260810 22:58:53.677881 18842 catalog_manager.cc:1259] Loaded cluster ID: 4f2c2286fd894f7b89178f1c114c0b7c
I20260810 22:58:53.677958 18842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: loading cluster ID for follower catalog manager: success
I20260810 22:58:53.679816 18842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: acquiring CA information for follower catalog manager: success
I20260810 22:58:53.680155 18842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 22:58:53.681825 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.673981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.6.47.189:38843)
0810 22:58:53.674180 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.674181 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.674188 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:53.674208 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.674208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.674217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.674297 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.674987 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.676061 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.676993 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.677313 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.677389 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.678714 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.678715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.678715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.678722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.680642 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.680994 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.680996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.680996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.681062 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.681283 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.681284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.681284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.681402 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.681509 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.681620 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.681707 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.6.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.188.keytab.
Entry for principal oryx/127.6.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.188.keytab.
I20260810 22:58:53.709751 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:41407
--webserver_interface=127.6.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.188.keytab
--principal=oryx/127.6.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:53.796746 18846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:53.796952 18846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:53.796973 18846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:53.798053 18846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:53.798091 18846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:53.798105 18846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:53.798117 18846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:53.798128 18846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:53.799557 18846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.188.keytab
--principal=oryx/127.6.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:41407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:53.799849 18846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:53.800072 18846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.804925 18856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.804975 18853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.804942 18854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.805282 18846 server_base.cc:1034] running on GCE node
I20260810 22:58:53.805560 18846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.805876 18846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.807072 18846 hybrid_clock.cc:648] HybridClock initialized: now 1786402733807046 us; error 52 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.809798 18846 init.cc:377] Logged in from keytab as oryx/127.6.47.188@KRBTEST.COM (short username oryx)
I20260810 22:58:53.810248 18846 webserver.cc:460] Webserver started at http://127.6.47.188:46409/ using document root <none> and password file <none>
I20260810 22:58:53.810513 18846 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.810582 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.810734 18846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:53.811785 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/instance:
uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "2077ba070861b72e566f96fc7c255a44"
server_key_iv: "ab83e0e733165b737e3b8af8fa58da9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.812184 18846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal/instance:
uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "2077ba070861b72e566f96fc7c255a44"
server_key_iv: "ab83e0e733165b737e3b8af8fa58da9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.813633 18846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:53.814366 18867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.814540 18846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:53.814633 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "2077ba070861b72e566f96fc7c255a44"
server_key_iv: "ab83e0e733165b737e3b8af8fa58da9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.814720 18846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.824625 18846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.826933 18846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:53.827096 18846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.831163 18846 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.188:41407
I20260810 22:58:53.831216 18963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.188:41407 every 8 connection(s)
I20260810 22:58:53.831589 18846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
I20260810 22:58:53.832721 18966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:53.834335 18966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:58:53.836771 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 18846
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.188@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:53.844337 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.835225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.6.47.190:36187)
0810 22:58:53.835306 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.835308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.835390 (+ 82us) server_negotiation.cc:407] Connection header received
0810 22:58:53.835846 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.835847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.835858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.835927 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.836594 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.837140 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.838649 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.838937 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.839007 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.841507 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.841508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.841509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.841517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.842895 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.843290 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.843291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.843292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.843356 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.843649 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.843651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.843651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.843784 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.843865 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.844145 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.844248 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 22:58:53.844559 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.835050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42322)
0810 22:58:53.835266 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.835310 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.835760 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.836006 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.836009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.836251 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.836471 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.836475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.837241 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.837242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.838570 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.838573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.839018 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.839019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.839155 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.839678 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.839688 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.841407 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.842980 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.842987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.842988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.843209 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.843444 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.843445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.843445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.843569 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.843858 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.843865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.844001 (+ 136us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.844114 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.844281 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":111,"threads_started":1}
I20260810 22:58:53.845345 18966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } has no permanent_uuid. Determining permanent_uuid...
I20260810 22:58:53.846838 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.837430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51390 (local address 127.6.47.188:41407)
0810 22:58:53.837626 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.837630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.837653 (+ 23us) server_negotiation.cc:407] Connection header received
0810 22:58:53.837680 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.837681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.837698 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.837760 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.838472 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.839425 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.840484 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.841017 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.841155 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.843738 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.843759 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.843759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.843765 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.845711 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.846088 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.846091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.846092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.846153 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.846362 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.846362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.846363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.846490 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.846617 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.846713 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.846771 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.188@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
I20260810 22:58:53.852123 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.845623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41674)
0810 22:58:53.845688 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.845700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.845745 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.845895 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.845895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.845997 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.846025 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.846026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.846943 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.846944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.847579 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.847582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.847994 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.847995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.848052 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.848378 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.848381 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.849647 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.851048 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.851049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.851050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.851259 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.851480 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.851481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.851482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.851538 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.851799 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.851800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.851840 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.851923 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.852037 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 22:58:53.852147 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.845694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.6.47.189:38843)
0810 22:58:53.845740 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.845742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.845752 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:53.845814 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.845816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.845829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.845889 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.846150 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.846870 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.847681 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.847893 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.847983 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.849739 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.849740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.849741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.849748 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.850965 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.851349 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.851350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.851351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.851406 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.851609 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.851611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.851611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.851724 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.851796 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.851948 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.852047 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:53.852741 18966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.188@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:53.858215 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.852987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:51396)
0810 22:58:53.853035 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.853046 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.853083 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.853206 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.853207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.853310 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.853336 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.853337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.853848 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.853849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.854307 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.854308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.854635 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.854636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.854672 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.855067 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.855072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.856121 (+ 1049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.857414 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.857415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.857415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.857579 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.857763 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.857764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.857764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.857816 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.858010 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.858010 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.858042 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.858091 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.858159 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 22:58:53.858278 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.853036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.6.47.188:41407)
0810 22:58:53.853068 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.853070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.853076 (+ 6us) server_negotiation.cc:407] Connection header received
0810 22:58:53.853129 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.853129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.853137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.853179 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.853373 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.853789 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.854375 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.854571 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.854631 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.856195 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.856196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.856197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.856202 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.857346 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.857655 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.857656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.857656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.857703 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.857869 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.857869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.857869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.857958 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.858005 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.858106 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.858202 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":17}
I20260810 22:58:53.859721 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Bootstrap starting.
I20260810 22:58:53.860569 18966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:53.860881 18966 log.cc:824] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:53.861629 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: No bootstrap required, opened a new log
I20260810 22:58:53.863026 18966 raft_consensus.cc:348] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } }
I20260810 22:58:53.863174 18966 raft_consensus.cc:374] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:53.863229 18966 raft_consensus.cc:729] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f72ed3a6d87f412883aedbc97c3b6a9a, State: Initialized, Role: FOLLOWER
I20260810 22:58:53.863363 18966 consensus_queue.cc:260] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } }
I20260810 22:58:53.863672 18966 sys_catalog.cc:564] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:53.863641 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } } }
I20260810 22:58:53.863765 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: FOLLOWER
W20260810 22:58:53.865643 18990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 22:58:53.865746 18990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 22:58:53.868554 18846 master_runner.cc:428] Detected that this master f72ed3a6d87f412883aedbc97c3b6a9a is joining an existing cluster
I20260810 22:58:53.868605 18846 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.47.188:41407
I20260810 22:58:53.877233 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.6.47.190:36187)
0810 22:58:53.869724 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.869726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.870123 (+ 397us) server_negotiation.cc:407] Connection header received
0810 22:58:53.870331 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.870333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.870343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.870392 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.870998 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871753 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.872902 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873223 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873266 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.874364 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.874365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.874365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.874371 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.875917 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.876348 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.876349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.876349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.876431 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.876691 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.876692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.876692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.876806 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.876877 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.877063 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.877146 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:53.877254 19003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42338)
0810 22:58:53.870031 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.870043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.870124 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.870547 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.870548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.870828 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.870898 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.870900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871854 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.871855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.872798 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.872801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873345 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.873431 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.873828 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.873833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.874267 (+ 434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.876036 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.876037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.876037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.876273 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.876538 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.876539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.876539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.876609 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.876880 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.876882 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.876964 (+ 82us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.877036 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.877146 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402732, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.190@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:53.896826 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41680)
0810 22:58:53.870050 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.870061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.870131 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.870437 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.870438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.870798 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.870889 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.870891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871786 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.871787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.872777 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.872780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873296 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.873386 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.893021 (+ 19635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.893027 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.893517 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.895503 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.895504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.895505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.895884 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.896094 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.896095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.896167 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.896447 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.896448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.896529 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.896593 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.896705 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":117,"thread_start_us":102,"threads_started":1}
I20260810 22:58:53.897266 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.6.47.189:38843)
0810 22:58:53.869875 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.869877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.870107 (+ 230us) server_negotiation.cc:407] Connection header received
0810 22:58:53.870214 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.870215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.870229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.870291 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.870986 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871698 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.872876 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873162 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873221 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.893622 (+ 20401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.893624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.893624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.893633 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.895402 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.895966 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.895967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.895968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.896028 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896233 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.896234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.896234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.896365 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.896423 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.897058 (+ 635us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.897173 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:58:53.897728 18969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.6.47.188:41407)
0810 22:58:53.869957 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.869959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.870306 (+ 347us) server_negotiation.cc:407] Connection header received
0810 22:58:53.870368 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.870369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.870378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.870432 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.870978 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871695 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.872872 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873158 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873211 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.893810 (+ 20599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.893811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.893812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.893819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.896336 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896716 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.896716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.896717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.896764 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896986 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.896987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.896988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.897098 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.897143 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.897340 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.897595 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":167,"server-negotiator.queue_time_us":19}
I20260810 22:58:53.897934 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.869906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:51412)
0810 22:58:53.870238 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.870246 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.870314 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.870437 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.870438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.870806 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.870889 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.870891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.871786 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.871787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.872777 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.872780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.873295 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.873296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.873365 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.893197 (+ 19832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.893203 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.893713 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.896416 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.896417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.896642 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.896832 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.896833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.896914 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.897180 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.897180 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.897234 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.897716 (+ 482us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.897820 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":84,"thread_start_us":66,"threads_started":1}
I20260810 22:58:53.898763 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 17754
I20260810 22:58:53.899379 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.879368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:51418)
0810 22:58:53.879421 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.879432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.879471 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.879737 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.879738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.879887 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.879921 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.879922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.880580 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.880582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.881107 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.881109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.881496 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.881497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.881574 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.881998 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.882003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.883403 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.896230 (+ 12827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.896232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.896469 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.896736 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.896738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.896806 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.898997 (+ 2191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.898998 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.899053 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.899102 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.899231 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 22:58:53.900153 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.879436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.6.47.188:41407)
0810 22:58:53.879607 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.879608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.879614 (+ 6us) server_negotiation.cc:407] Connection header received
0810 22:58:53.879645 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.879645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.879654 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.879719 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.880003 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.880465 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.881198 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.881422 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.881497 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.883482 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.883483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.883483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.883489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.896114 (+ 12625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.896587 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.896588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.896589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.896641 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.897224 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.897225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.897225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.898907 (+ 1682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.899833 (+ 926us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.899887 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.899991 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260810 22:58:53.914328 17612 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.47.188:41407: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
member_type: NON_VOTER
last_known_addr {
host: "127.6.47.188"
port: 41407
}
attrs {
promote: true
}
}
dest_uuid: "8b28120137c2490091fa71c46ecfc396"
cas_config_opid_index: 5
I20260810 22:58:53.914585 17612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
I20260810 22:58:53.915350 18917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:53.915588 18685 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.916440 18917 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 22:58:53.917003 18686 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:53.917399 18685 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.47.188:41407
I20260810 22:58:53.917559 18686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.917650 18686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:53.918918 18846 master.cc:537] Master@127.6.47.188:41407 shutting down...
I20260810 22:58:53.920192 18917 raft_consensus.cc:858] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
W20260810 22:58:53.920259 18917 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
I20260810 22:58:53.920974 18917 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cba40bc5ebbe479799858da66d24f307 (127.6.47.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.922341 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b28120137c2490091fa71c46ecfc396. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.922474 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 22:58:53.922528 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.922600 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 22:58:53.922639 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } }
I20260810 22:58:53.922698 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 22:58:53.922786 19008 mvcc.cc:205] Tried to move back new op lower bound from 7317105594035949568 to 7317105593987067904. Current Snapshot: MvccSnapshot[applied={T|T < 7317105593998082048}]
I20260810 22:58:53.923910 18686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.923993 18686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
W20260810 22:58:53.924525 17586 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 -> Peer f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188:41407): Couldn't send request to peer f72ed3a6d87f412883aedbc97c3b6a9a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 22:58:53.924782 18846 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 22:58:53.924888 18846 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 22:58:53.924924 18846 tablet_replica.cc:330] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: stopping tablet replica
I20260810 22:58:53.925318 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:34731
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:36187
--webserver_port=45741
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:58:53.925493 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.918318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:41692)
0810 22:58:53.918651 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.918674 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.918727 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.919399 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.919401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.919403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:53.919458 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.919459 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.920768 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.920770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.924087 (+ 3317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.924091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.925229 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.925231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.925298 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.925323 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.925376 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.925421 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305}
I20260810 22:58:53.925745 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.918549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.6.47.189:38843)
0810 22:58:53.919168 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.919170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.919180 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:53.919208 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.919210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.919233 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.919317 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:53.919530 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.920291 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.924194 (+ 3903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.925033 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.925114 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.925191 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.925635 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.925680 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588}
I20260810 22:58:53.925932 18755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 LEARNER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 22:58:53.926122 18685 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 22:58:53.926636 18755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.927721 18828 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b28120137c2490091fa71c46ecfc396. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.927817 18828 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: LEARNER
I20260810 22:58:53.927865 18828 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: NON_VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: true } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.927901 18828 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: LEARNER
I20260810 22:58:53.928382 18685 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: attempting to promote NON_VOTER cba40bc5ebbe479799858da66d24f307 to VOTER
I20260810 22:58:53.928586 18685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
I20260810 22:58:53.929024 18755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 LEARNER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 22:58:53.929494 19007 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 22:58:53.929576 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.929195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:51432)
0810 22:58:53.929449 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.929482 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:41407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260810 22:58:53.930065 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.929936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:51438)
0810 22:58:53.929997 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.930032 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:41407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:58:53.930193 17586 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 -> Peer f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188:41407): Couldn't send request to peer f72ed3a6d87f412883aedbc97c3b6a9a. Status: Network error: Client connection negotiation failed: client connection to 127.6.47.188:41407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:58:53.930269 19007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cba40bc5ebbe479799858da66d24f307 (127.6.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.930428 18755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cba40bc5ebbe479799858da66d24f307 (127.6.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.931324 19007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.931408 19007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:53.931457 19031 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.931535 19031 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:58:53.931478 19007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.931564 19007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:53.940312 18846 master.cc:552] Master@127.6.47.188:41407 shutdown complete.
I20260810 22:58:53.940369 18846 master_runner.cc:305] Clearing existing system tablet
I20260810 22:58:53.940522 18846 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.940575 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.941293 18846 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.941851 19035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.941965 18846 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.942013 18846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.942052 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "2077ba070861b72e566f96fc7c255a44"
server_key_iv: "ab83e0e733165b737e3b8af8fa58da9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.942764 18846 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:53.944159 18846 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 22:58:53.944233 18846 log.cc:1196] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 22:58:53.944463 18846 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Deleting consensus metadata
I20260810 22:58:53.944615 18846 master_runner.cc:315] Copying system tablet from 127.6.47.190:36187
I20260810 22:58:53.945233 18846 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: tablet copy: Beginning tablet copy session from remote peer at address 127.6.47.190:36187
I20260810 22:58:53.952781 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.945584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.6.47.190:36187)
0810 22:58:53.945639 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:53.945641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:53.945764 (+ 123us) server_negotiation.cc:407] Connection header received
0810 22:58:53.945815 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:53.945816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:53.945826 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:53.945886 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:53.946253 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.947023 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.948059 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.948370 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.948444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.949736 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:53.949738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:53.949738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:53.949747 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:53.951468 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.951855 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.951856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.951856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.951920 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:53.952174 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:53.952174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:53.952175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:53.952306 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:53.952374 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:53.952618 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 22:58:53.952696 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:58:53.952800 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:53.945504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:42352)
0810 22:58:53.945670 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:53.945683 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:53.945761 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:53.945892 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:53.945893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:53.946127 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:53.946172 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.946174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.947130 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.947131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.947959 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:53.947963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:53.948467 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:53.948469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:53.948555 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:53.949118 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:53.949124 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:53.949622 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:53.951558 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.951559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.951560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.951776 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.951995 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:53.951996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:53.951996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:53.952089 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:53.952381 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:53.952382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:53.952437 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 22:58:53.952588 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 22:58:53.952696 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260810 22:58:53.953522 17656 tablet_copy_service.cc:140] P 8b28120137c2490091fa71c46ecfc396: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f72ed3a6d87f412883aedbc97c3b6a9a ({username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:42352)
I20260810 22:58:53.953584 17656 tablet_copy_service.cc:161] P 8b28120137c2490091fa71c46ecfc396: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f72ed3a6d87f412883aedbc97c3b6a9a at {username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:42352: session id = f72ed3a6d87f412883aedbc97c3b6a9a-00000000000000000000000000000000
I20260810 22:58:53.954134 17656 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 22:58:53.954631 18846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 22:58:53.955905 18846 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: tablet copy: Starting download of 0 data blocks...
I20260810 22:58:53.956040 18846 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: tablet copy: Starting download of 1 WAL segments...
I20260810 22:58:53.956933 18846 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 22:58:53.957880 17656 tablet_copy_service.cc:342] P 8b28120137c2490091fa71c46ecfc396: Request end of tablet copy session f72ed3a6d87f412883aedbc97c3b6a9a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:42352
I20260810 22:58:53.957937 17656 tablet_copy_service.cc:433] P 8b28120137c2490091fa71c46ecfc396: ending tablet copy session f72ed3a6d87f412883aedbc97c3b6a9a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f72ed3a6d87f412883aedbc97c3b6a9a
I20260810 22:58:53.961114 18846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:53.962458 19051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.962491 19052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:53.962623 19054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:53.963204 18846 server_base.cc:1034] running on GCE node
I20260810 22:58:53.963310 18846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:53.963436 18846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:53.964546 18846 hybrid_clock.cc:648] HybridClock initialized: now 1786402733964523 us; error 38 us; skew 500 ppm
Aug 10 22:58:53 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402733, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:53.966070 18846 init.cc:377] Logged in from keytab as oryx/127.6.47.188@KRBTEST.COM (short username oryx)
I20260810 22:58:53.966346 18846 webserver.cc:460] Webserver started at http://127.6.47.188:36947/ using document root <none> and password file <none>
I20260810 22:58:53.966460 18846 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:53.966493 18846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:53.967198 18846 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.967784 19071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:53.967988 18846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:53.968156 18846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "f72ed3a6d87f412883aedbc97c3b6a9a"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "2077ba070861b72e566f96fc7c255a44"
server_key_iv: "ab83e0e733165b737e3b8af8fa58da9a"
server_key_version: "encryptionkey@0"
I20260810 22:58:53.968276 18846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:53.982329 18846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:53.984454 18846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:53.987713 18846 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.188:41407
I20260810 22:58:53.987989 19170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.188:41407 every 8 connection(s)
I20260810 22:58:53.988267 18846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
I20260810 22:58:53.989302 19171 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:58:53.989799 19171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Bootstrap starting.
I20260810 22:58:53.992177 19171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:58:53.992395 19171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Bootstrap complete.
I20260810 22:58:53.992635 19171 raft_consensus.cc:348] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
I20260810 22:58:53.992731 19171 raft_consensus.cc:729] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f72ed3a6d87f412883aedbc97c3b6a9a, State: Initialized, Role: LEARNER
I20260810 22:58:53.992771 19171 consensus_queue.cc:260] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } }
I20260810 22:58:53.992969 19171 sys_catalog.cc:564] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:53.992975 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:53.993213 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: LEARNER
I20260810 22:58:53.994462 19195 catalog_manager.cc:1259] Loaded cluster ID: 4f2c2286fd894f7b89178f1c114c0b7c
I20260810 22:58:53.994673 19195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: loading cluster ID for follower catalog manager: success
I20260810 22:58:53.996088 19195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: acquiring CA information for follower catalog manager: success
I20260810 22:58:53.996479 19195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 22:58:54.028093 19027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:54.028368 19027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:54.028405 19027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:54.030572 19027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:54.030627 19027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:54.030655 19027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:54.033046 19027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:34731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=45741
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:54.033344 19027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:54.033589 19027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:54.040247 19210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:54.040202 19201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:54.040390 19204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:54.045759 19027 server_base.cc:1034] running on GCE node
I20260810 22:58:54.046129 19027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:54.046479 19027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:54.047689 19027 hybrid_clock.cc:648] HybridClock initialized: now 1786402734047651 us; error 36 us; skew 500 ppm
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:54.051141 19027 init.cc:377] Logged in from keytab as oryx/127.6.47.129@KRBTEST.COM (short username oryx)
I20260810 22:58:54.051669 19027 webserver.cc:460] Webserver started at http://127.6.47.129:45741/ using document root <none> and password file <none>
I20260810 22:58:54.051901 19027 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:54.051995 19027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:54.053869 19027 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:54.054692 19223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.054878 19027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 22:58:54.054999 19027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "c5a8217a27684e27aea8fd642e533813"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "e8538b6aa0d368bbf3d9424f649a7051"
server_key_iv: "7b9a918d2c972fc039f3573058ef8be4"
server_key_version: "encryptionkey@0"
I20260810 22:58:54.055548 19027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:54.076864 19027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:54.078728 19027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:54.078958 19027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:54.079228 19027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:54.079571 19027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:54.079630 19027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.079675 19027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:54.079702 19027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.086364 19027 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:34731
I20260810 22:58:54.086413 19436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:34731 every 8 connection(s)
I20260810 22:58:54.086740 19027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:54.093400 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 19027
I20260810 22:58:54.093569 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 18001
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:54.103950 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50383 (local address 127.6.47.190:36187)
0810 22:58:54.087671 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.087673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.087857 (+ 184us) server_negotiation.cc:407] Connection header received
0810 22:58:54.089066 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.089068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.089082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.089153 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.090045 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.090554 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.091761 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.091988 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.092051 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.095424 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.095425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.095425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.095433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.102370 (+ 6937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.102894 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.102895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.102896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.102951 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.103198 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.103199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.103200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.103329 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.103369 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.103735 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.103820 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260810 22:58:54.104471 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:50383)
0810 22:58:54.087697 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.087745 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.088954 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.089227 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.089231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.089658 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.089858 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.089863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.090652 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.090653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.091689 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.091692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.092117 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.092118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.092171 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.092785 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.092810 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.095307 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.102468 (+ 7161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.102478 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.102480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.102813 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.103012 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.103013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.103014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.103137 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.103406 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.103412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.103566 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.104050 (+ 484us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.104224 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":29,"thread_start_us":94,"threads_started":1}
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
I20260810 22:58:54.108675 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.6.47.129:52921)
0810 22:58:54.087836 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.087845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.088967 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.089457 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.089459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.089683 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.089859 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.089865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.091687 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.091687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.092316 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.092319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.093146 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.093147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.093217 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.093673 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.093678 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.095307 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.107006 (+ 11699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.107007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.107008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.107263 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.107673 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.107674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.107675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.107764 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.108170 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.108171 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.108229 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.108306 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.108458 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260810 22:58:54.109169 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:52921 (local address 127.6.47.188:41407)
0810 22:58:54.087650 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.087652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.087924 (+ 272us) server_negotiation.cc:407] Connection header received
0810 22:58:54.089354 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.089355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.089368 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.089438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.090573 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.091614 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.092435 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.092951 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.093010 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.095418 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.095420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.095420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.095427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.106863 (+ 11436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.107469 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.107471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.107472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.107553 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.107854 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.107855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.107856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.108044 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.108136 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.108899 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.109024 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260810 22:58:54.109701 19438 heartbeater.cc:347] Connected to a master server at 127.6.47.188:41407
I20260810 22:58:54.109836 19438 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.110124 19438 heartbeater.cc:511] Master 127.6.47.188:41407 requested a full tablet report, sending...
I20260810 22:58:54.110813 19104 ts_manager.cc:194] Registered new tserver with Master: c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731)
I20260810 22:58:54.111325 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:37249 (local address 127.6.47.189:38843)
0810 22:58:54.087627 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.087629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.087902 (+ 273us) server_negotiation.cc:407] Connection header received
0810 22:58:54.089213 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.089214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.089226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.089295 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.089966 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.090483 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.091613 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.091896 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.091970 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.108413 (+ 16443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.108415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.108415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.108423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.110210 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.110574 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.110575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.110575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.110619 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.110808 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.110809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.110809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.110938 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.110973 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.111128 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.111205 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:54.111687 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.087413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.6.47.129:37249)
0810 22:58:54.087825 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.087837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.088967 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.089318 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.089319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.089694 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.089859 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.089865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.090602 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.090603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.091519 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.091522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.091994 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.091995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.092137 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.092846 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.092851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.108311 (+ 15460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.110301 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.110302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.110302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.110510 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.110695 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.110695 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.110696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.110750 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.111007 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.111008 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.111043 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.111429 (+ 386us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.111532 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":87,"thread_start_us":286,"threads_started":1}
I20260810 22:58:54.113150 19439 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:54.113214 19439 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.113304 19439 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:54.114416 17612 ts_manager.cc:194] Re-registered known tserver with Master: c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731)
I20260810 22:58:54.115120 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:50383
I20260810 22:58:54.119263 19440 heartbeater.cc:347] Connected to a master server at 127.6.47.189:38843
I20260810 22:58:54.119318 19440 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.119397 19440 heartbeater.cc:511] Master 127.6.47.189:38843 requested a full tablet report, sending...
I20260810 22:58:54.119855 18751 ts_manager.cc:194] Registered new tserver with Master: c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731)
I20260810 22:58:54.123445 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:45555
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:36187
--webserver_port=41579
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:54.209266 19448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:54.209488 19448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:54.209511 19448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:54.210661 19448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:54.210704 19448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:54.210754 19448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:54.212268 19448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:45555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=41579
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:54.212548 19448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:54.212803 19448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:54.217736 19459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:54.217778 19448 server_base.cc:1034] running on GCE node
W20260810 22:58:54.217751 19460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:54.217757 19466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:54.218192 19448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:54.218413 19448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:54.219585 19448 hybrid_clock.cc:648] HybridClock initialized: now 1786402734219527 us; error 112 us; skew 500 ppm
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:54.222163 19448 init.cc:377] Logged in from keytab as oryx/127.6.47.130@KRBTEST.COM (short username oryx)
I20260810 22:58:54.222666 19448 webserver.cc:460] Webserver started at http://127.6.47.130:41579/ using document root <none> and password file <none>
I20260810 22:58:54.222901 19448 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:54.222988 19448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:54.224367 19448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:54.225013 19484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.225204 19448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.225304 19448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "39fcf34c07912e35815d47dadb1309a7"
server_key_iv: "ec8bec2db6069875b98524b26cd6f63a"
server_key_version: "encryptionkey@0"
I20260810 22:58:54.225627 19448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:54.242213 19448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:54.244007 19448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:54.244181 19448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:54.244413 19448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:54.244764 19448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:54.244820 19448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.244855 19448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:54.244882 19448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.251135 19448 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:45555
I20260810 22:58:54.251180 19691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:45555 every 8 connection(s)
I20260810 22:58:54.251523 19448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:54.259635 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 19448
I20260810 22:58:54.260085 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 18259
I20260810 22:58:54.263628 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:59509 (local address 127.6.47.190:36187)
0810 22:58:54.252778 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.252781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.252791 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:54.253649 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.253650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.253662 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.253703 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.254370 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.255109 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256385 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256682 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256754 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.259809 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.259811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.259811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.259818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.261959 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.262438 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.262439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.262439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.262513 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.262785 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.262786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.262786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.262960 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.263044 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.263360 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.263480 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 22:58:54.264022 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:59509)
0810 22:58:54.252736 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.252747 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.253523 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.253791 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.253792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.254145 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.254262 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.254264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.255549 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.255549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.256292 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.256294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256809 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.256869 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.257373 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.257377 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.259728 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.262054 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.262061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.262063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.262351 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.262589 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.262590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.262590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.262715 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.263037 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.263041 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.263198 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.263312 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.263494 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":22,"thread_start_us":89,"threads_started":1}
I20260810 22:58:54.265978 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:38323 (local address 127.6.47.188:41407)
0810 22:58:54.252587 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.252589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.252601 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:58:54.253646 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.253648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.253663 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.253696 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.254370 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.255100 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256083 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256375 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256437 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.260180 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.260181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.260181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.260189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.264712 (+ 4523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.265074 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.265075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.265076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.265147 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.265352 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.265353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.265354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.265513 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.265582 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.265737 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.265847 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330}
I20260810 22:58:54.265995 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.6.47.130:38323)
0810 22:58:54.252390 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.252434 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.253517 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.253790 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.253793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.254142 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.254262 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.254264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.255183 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.255184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.255989 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.255991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256440 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.256602 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.257177 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.257185 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.260070 (+ 2885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.264806 (+ 4736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.264807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.264808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.265001 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.265229 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.265230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.265230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.265284 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.265581 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.265582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.265625 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.265697 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.265821 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":56,"thread_start_us":100,"threads_started":1}
I20260810 22:58:54.267580 19692 heartbeater.cc:347] Connected to a master server at 127.6.47.188:41407
I20260810 22:58:54.267697 19692 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.267949 19692 heartbeater.cc:511] Master 127.6.47.188:41407 requested a full tablet report, sending...
I20260810 22:58:54.268469 19104 ts_manager.cc:194] Registered new tserver with Master: f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:54.275360 19694 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:54.275424 19694 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.275528 19694 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:54.275870 17612 ts_manager.cc:194] Re-registered known tserver with Master: f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:54.276455 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:59509
I20260810 22:58:54.280469 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:41759
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:36187
--webserver_port=44245
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:58:54.281520 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:34295 (local address 127.6.47.189:38843)
0810 22:58:54.252327 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.252330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.252783 (+ 453us) server_negotiation.cc:407] Connection header received
0810 22:58:54.253631 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.253632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.253644 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.253706 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.254357 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.254844 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256155 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256459 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256554 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.258926 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.258927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.258928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.258936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.260721 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.280627 (+ 19906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.280629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.280630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.280712 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.280915 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.280916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.280916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.281073 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.281149 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.281282 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.281379 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:54.281553 19701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.252215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.6.47.130:34295)
0810 22:58:54.252716 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.252726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.253528 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.253789 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.253792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.254068 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.254260 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.254264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.254957 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.254958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.256062 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.256064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.256566 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.256568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.256634 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.257211 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.257216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.258836 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.280257 (+ 21421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.280258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.280259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.280488 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.280785 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.280786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.280786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.280846 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.281146 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.281147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.281183 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.281236 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.281332 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260810 22:58:54.282200 19696 heartbeater.cc:347] Connected to a master server at 127.6.47.189:38843
I20260810 22:58:54.282272 19696 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.282384 19696 heartbeater.cc:511] Master 127.6.47.189:38843 requested a full tablet report, sending...
I20260810 22:58:54.282689 18751 ts_manager.cc:194] Registered new tserver with Master: f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
W20260810 22:58:54.365800 19706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:54.366003 19706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:54.366025 19706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:54.367131 19706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:54.367175 19706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:54.367201 19706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:54.368791 19706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:41759
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=44245
--tserver_master_addrs=127.6.47.190:36187,127.6.47.189:38843,127.6.47.188:41407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:54.369050 19706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:54.369381 19706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:54.374661 19718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:54.374725 19714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:54.374661 19715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:54.374900 19706 server_base.cc:1034] running on GCE node
I20260810 22:58:54.375118 19706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:54.375379 19706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:54.376540 19706 hybrid_clock.cc:648] HybridClock initialized: now 1786402734376529 us; error 29 us; skew 500 ppm
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:54.379033 19706 init.cc:377] Logged in from keytab as oryx/127.6.47.131@KRBTEST.COM (short username oryx)
I20260810 22:58:54.379527 19706 webserver.cc:460] Webserver started at http://127.6.47.131:44245/ using document root <none> and password file <none>
I20260810 22:58:54.379789 19706 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:54.379873 19706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:54.381309 19706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:54.381893 19738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.382071 19706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:54.382175 19706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "1c93871cf3e04ce2b9febf227378ed43"
format_stamp: "Formatted at 2026-08-10 22:58:53 on dist-test-slave-rwrg"
server_key: "3f991d0a64d372a664da7a7d8e547369"
server_key_iv: "1d15604f22b7b5e42f44c40a49627299"
server_key_version: "encryptionkey@0"
I20260810 22:58:54.382511 19706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:54.431097 19706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:54.433555 19706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:54.433730 19706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:54.433980 19706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:58:54.434314 19706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:54.434371 19706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.434418 19706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:54.434454 19706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:54.440913 19706 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:41759
I20260810 22:58:54.440963 19926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:41759 every 8 connection(s)
I20260810 22:58:54.441326 19706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:54.446822 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 19706
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.189@KRBTEST.COM
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
Aug 10 22:58:54 dist-test-slave-rwrg krb5kdc[17525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402734, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.188@KRBTEST.COM
I20260810 22:58:54.456487 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.6.47.131:47437)
0810 22:58:54.442351 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.442393 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.442973 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.443293 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.443297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.443801 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.444064 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.444070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.445048 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.445049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.446015 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.446017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446533 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.446617 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.449832 (+ 3215us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.449837 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.451260 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.454814 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.454815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.454816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.455053 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.455261 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.455262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.455263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.455398 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.455703 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.455711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.455884 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.455994 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.456205 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":67,"thread_start_us":93,"threads_started":1}
I20260810 22:58:54.456799 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.447960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.6.47.188:41407)
0810 22:58:54.448241 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.448243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.448417 (+ 174us) server_negotiation.cc:407] Connection header received
0810 22:58:54.448439 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.448440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.448450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.448556 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.449616 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.450407 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.451579 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.451865 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.451905 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.453392 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.453394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.453394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.453401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.455504 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.455862 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.455863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.455863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.455918 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.456134 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.456135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.456136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.456283 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.456357 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.456618 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.456708 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":68,"threads_started":1}
I20260810 22:58:54.456970 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:47437 (local address 127.6.47.188:41407)
0810 22:58:54.442291 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.442293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.442481 (+ 188us) server_negotiation.cc:407] Connection header received
0810 22:58:54.443102 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.443112 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.443125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.443157 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.444185 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.444950 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446102 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446403 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446506 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.451626 (+ 5120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.451627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.451627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.451634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.454749 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.455137 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.455139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.455139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.455191 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.455474 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.455475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.455476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.455617 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.455676 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.456825 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.456899 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 22:58:54.457361 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:43287 (local address 127.6.47.190:36187)
0810 22:58:54.442252 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.442254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.442481 (+ 227us) server_negotiation.cc:407] Connection header received
0810 22:58:54.443102 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.443103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.443118 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.443154 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.444181 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.444950 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446137 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446395 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446465 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.449758 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.449759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.449760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.449767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.451525 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.455870 (+ 4345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.455871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.455872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.455925 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.456157 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.456159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.456159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.456360 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.456445 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.457177 (+ 732us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.457254 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 22:58:54.457883 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.448175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.6.47.190:36187)
0810 22:58:54.448290 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.448292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.448299 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:54.448321 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.448322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.448336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.448418 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.449624 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.450400 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.451877 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.452226 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.452595 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.453729 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.453731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.453731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.453738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.455689 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457049 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.457050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.457050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.457117 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457342 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.457343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.457343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.457467 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.457585 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.457689 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.457765 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260810 22:58:54.457090 19934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:43287)
0810 22:58:54.442351 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.442393 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.442973 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.443293 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.443297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.443799 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.444064 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.444070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.445048 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.445049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.446012 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.446014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446507 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.446595 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.447386 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.447391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.449665 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.452195 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.452203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.452206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.453046 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.456001 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.456002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.456003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.456081 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.456476 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.456477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.456543 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.456624 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.456727 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":107,"spinlock_wait_cycles":35072,"thread_start_us":112,"threads_started":1}
I20260810 22:58:54.458420 19930 heartbeater.cc:347] Connected to a master server at 127.6.47.190:36187
I20260810 22:58:54.458510 19928 heartbeater.cc:347] Connected to a master server at 127.6.47.188:41407
I20260810 22:58:54.458561 19928 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.458774 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:58793 (local address 127.6.47.189:38843)
0810 22:58:54.442445 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.442447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.442555 (+ 108us) server_negotiation.cc:407] Connection header received
0810 22:58:54.443102 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.443104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.443117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.443179 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.444174 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.444792 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.445901 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446179 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446252 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.451625 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.451626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.451627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.451634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.455784 (+ 4150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457712 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.457713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.457714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.457781 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457994 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.457995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.457995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.458144 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.458199 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.458617 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.458693 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:54.458734 19935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.442102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.6.47.131:58793)
0810 22:58:54.442491 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.442504 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.442988 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.443634 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.443635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.443865 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:54.444062 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.444067 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.444911 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.444912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.445801 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.445802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.446275 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.446277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.446416 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.447051 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:54.447063 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:54.451543 (+ 4480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:54.457413 (+ 5870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.457414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.457627 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.457858 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:54.457859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:54.457859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:54.457919 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:54.458350 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:54.458351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:54.458392 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.458460 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.458560 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":102,"threads_started":1}
I20260810 22:58:54.458765 19930 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.459043 19930 heartbeater.cc:511] Master 127.6.47.190:36187 requested a full tablet report, sending...
I20260810 22:58:54.458778 19928 heartbeater.cc:511] Master 127.6.47.188:41407 requested a full tablet report, sending...
I20260810 22:58:54.459452 19931 heartbeater.cc:347] Connected to a master server at 127.6.47.189:38843
I20260810 22:58:54.459503 19931 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:54.459496 19104 ts_manager.cc:194] Registered new tserver with Master: 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:54.459570 19931 heartbeater.cc:511] Master 127.6.47.189:38843 requested a full tablet report, sending...
I20260810 22:58:54.459849 18751 ts_manager.cc:194] Registered new tserver with Master: 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:54.460199 17612 ts_manager.cc:194] Re-registered known tserver with Master: 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:54.460723 17612 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:43287
I20260810 22:58:54.459481 17611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:54.462013 17611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:54.462839 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.448929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.6.47.189:38843)
0810 22:58:54.451657 (+ 2728us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.451659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.451666 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:54.451685 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.451686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.451697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.451760 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.452580 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.453381 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.454390 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.454720 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.459486 (+ 4766us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.459654 (+ 168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.459655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.459656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.459662 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.461519 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.461977 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.461978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.461978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.462061 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.462291 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.462292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.462293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.462422 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.462525 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.462656 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.462737 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2708,"thread_start_us":65,"threads_started":1}
I20260810 22:58:54.468204 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.464748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.6.47.188:41407)
0810 22:58:54.464786 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.464788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.464912 (+ 124us) server_negotiation.cc:407] Connection header received
0810 22:58:54.465030 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.465031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.465056 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.465104 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.465308 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.465963 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.467217 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.467786 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.467832 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.467909 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.468108 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.468142 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 22:58:54.468602 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.464673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:58800)
0810 22:58:54.464851 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.464860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.464937 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.465186 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.465186 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.465188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.465237 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.465238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.466076 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.466077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.467096 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.467101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.467972 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.467973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.468034 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.468037 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.468435 (+ 398us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.468496 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":138,"threads_started":1}
I20260810 22:58:54.469026 19129 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.8] Dedup: 1.7->[1.8-1.8]
I20260810 22:58:54.469449 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b28120137c2490091fa71c46ecfc396. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:54.469542 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: LEARNER
I20260810 22:58:54.470769 19969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:41759 (local address 127.0.0.1:60774)
0810 22:58:54.467086 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.467094 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.467130 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.467401 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.467401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.467403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.467437 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.467439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.468673 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.468674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.469800 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.469803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.470544 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.470545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.470603 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.470606 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.470669 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.470707 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260810 22:58:54.470788 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.6.47.131:41759)
0810 22:58:54.467248 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.467253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.467261 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:54.467293 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.467294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.467308 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.467501 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.467545 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.468565 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.469906 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.470447 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.470542 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.470620 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.470683 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.470713 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":72,"threads_started":1}
I20260810 22:58:54.471395 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34731 (local address 127.0.0.1:55766)
0810 22:58:54.466795 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.466803 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.466869 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.467140 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.467141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.467143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.467200 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.467203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.468590 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.468591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.469826 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.469828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.470797 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.470798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.471237 (+ 439us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.471241 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.471300 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.471333 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":51,"thread_start_us":103,"threads_started":1}
I20260810 22:58:54.471464 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.6.47.129:34731)
0810 22:58:54.466663 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.466668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.466864 (+ 196us) server_negotiation.cc:407] Connection header received
0810 22:58:54.466952 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.466955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.466974 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.467055 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.467279 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.468480 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.469908 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.470621 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.470688 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.470763 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.471333 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.471379 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260810 22:58:54.471441 19808 tablet_service.cc:1467] Processing CreateTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 (DEFAULT_TABLE table=test-table [id=9468b41ec8e2494e88cd781d64744af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:54.471846 19808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19fe7586a2cd45ae83c118cfb2afe5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:54.472169 19974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:45555 (local address 127.0.0.1:34546)
0810 22:58:54.466958 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.466967 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.467015 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.467264 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.467265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.467267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.467305 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.467306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.469610 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.469611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.470685 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.470688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.471873 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.471874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.471956 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.471960 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.472030 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.472086 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":68,"threads_started":1}
I20260810 22:58:54.472189 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.466312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.6.47.130:45555)
0810 22:58:54.466697 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.466702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.467035 (+ 333us) server_negotiation.cc:407] Connection header received
0810 22:58:54.467086 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.467089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.467106 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.467196 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.467376 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.468889 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.470976 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.471715 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.471775 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.471844 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.472074 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.472113 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":84,"threads_started":1}
I20260810 22:58:54.472503 19327 tablet_service.cc:1467] Processing CreateTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 (DEFAULT_TABLE table=test-table [id=9468b41ec8e2494e88cd781d64744af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:54.472748 19959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: NON_VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: true } } }
I20260810 22:58:54.472844 19959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:54.472961 19327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19fe7586a2cd45ae83c118cfb2afe5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:54.473199 19586 tablet_service.cc:1467] Processing CreateTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 (DEFAULT_TABLE table=test-table [id=9468b41ec8e2494e88cd781d64744af8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:54.473570 19586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19fe7586a2cd45ae83c118cfb2afe5a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:54.475489 19987 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Bootstrap starting.
I20260810 22:58:54.476265 19987 tablet_bootstrap.cc:654] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:54.476758 19989 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Bootstrap starting.
I20260810 22:58:54.476806 19987 log.cc:824] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:54.477476 19989 tablet_bootstrap.cc:654] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:54.477610 19962 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: attempting to promote NON_VOTER f72ed3a6d87f412883aedbc97c3b6a9a to VOTER
I20260810 22:58:54.477723 19987 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: No bootstrap required, opened a new log
I20260810 22:58:54.477784 19987 ts_tablet_manager.cc:1397] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:54.477763 19962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } }
I20260810 22:58:54.477834 19989 log.cc:824] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:54.478286 18755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260810 22:58:54.478348 19129 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 LEARNER]: Refusing update from remote peer 8b28120137c2490091fa71c46ecfc396: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260810 22:58:54.478545 19989 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: No bootstrap required, opened a new log
I20260810 22:58:54.478482 19959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260810 22:58:54.478617 19989 ts_tablet_manager.cc:1397] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:54.478606 19990 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Bootstrap starting.
I20260810 22:58:54.478682 19962 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260810 22:58:54.479274 19962 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.479406 19129 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.479615 19990 tablet_bootstrap.cc:654] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:54.479909 19990 log.cc:824] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:54.480268 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.480345 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:54.480396 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.480399 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.480459 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:54.480474 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:58:54.480687 18755 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.480660 19989 raft_consensus.cc:348] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.480813 19989 raft_consensus.cc:374] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:54.480856 19989 raft_consensus.cc:729] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a8217a27684e27aea8fd642e533813, State: Initialized, Role: FOLLOWER
I20260810 22:58:54.480917 19990 tablet_bootstrap.cc:492] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: No bootstrap required, opened a new log
I20260810 22:58:54.481000 19990 ts_tablet_manager.cc:1397] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:54.481032 19989 consensus_queue.cc:260] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.481505 19439 heartbeater.cc:503] Master 127.6.47.190:36187 was elected leader, sending a full tablet report...
I20260810 22:58:54.481760 19989 ts_tablet_manager.cc:1428] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:54.481966 19962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.482069 19962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:54.483645 19990 raft_consensus.cc:348] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.483803 19990 raft_consensus.cc:374] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:54.483835 19990 raft_consensus.cc:729] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a6fde08b5b4fda9389ee1c3d3a76ae, State: Initialized, Role: FOLLOWER
I20260810 22:58:54.483938 19990 consensus_queue.cc:260] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.484182 19990 ts_tablet_manager.cc:1428] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 22:58:54.484220 19694 heartbeater.cc:503] Master 127.6.47.190:36187 was elected leader, sending a full tablet report...
I20260810 22:58:54.484285 19987 raft_consensus.cc:348] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.484414 19987 raft_consensus.cc:374] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:54.484444 19987 raft_consensus.cc:729] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c93871cf3e04ce2b9febf227378ed43, State: Initialized, Role: FOLLOWER
I20260810 22:58:54.484571 19987 consensus_queue.cc:260] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.484820 19987 ts_tablet_manager.cc:1428] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260810 22:58:54.484910 19930 heartbeater.cc:503] Master 127.6.47.190:36187 was elected leader, sending a full tablet report...
I20260810 22:58:54.485881 19998 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b28120137c2490091fa71c46ecfc396" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:54.486091 19998 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 22:58:54.502354 19698 tablet.cc:2369] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:54.589195 19441 tablet.cc:2369] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:54.692206 19932 tablet.cc:2369] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:54.705024 19996 raft_consensus.cc:482] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:54.705173 19996 raft_consensus.cc:504] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.705513 19996 leader_election.cc:288] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555), 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:54.708693 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.705786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:51129 (local address 127.6.47.131:41759)
0810 22:58:54.705836 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.705839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.705888 (+ 49us) server_negotiation.cc:407] Connection header received
0810 22:58:54.705974 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.705975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.705986 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.706059 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.706209 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.706814 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.707934 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.708301 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.708382 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.708392 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.708571 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.708608 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:58:54.708694 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.705691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:41759 (local address 127.6.47.129:51129)
0810 22:58:54.705833 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.705845 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.705906 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.706065 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.706066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.706068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.706122 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.706124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.706894 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.706895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.707842 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.707844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.708382 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.708383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.708457 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.708460 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.708562 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.708611 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":20,"thread_start_us":62,"threads_started":1}
I20260810 22:58:54.708808 19972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.705781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:49051 (local address 127.6.47.130:45555)
0810 22:58:54.705846 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.705849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.705953 (+ 104us) server_negotiation.cc:407] Connection header received
0810 22:58:54.706036 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.706037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.706050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.706133 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:54.706322 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.706911 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.708011 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.708412 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.708527 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.708536 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.708700 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.708738 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 22:58:54.708820 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.705716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:45555 (local address 127.6.47.129:49051)
0810 22:58:54.705869 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:54.705880 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:54.705968 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:54.706137 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:54.706138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:54.706141 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:54.706226 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.706228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.707008 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.707010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.707896 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:54.707899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.708537 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.708539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:54.708618 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.708621 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:54.708694 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 22:58:54.708735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260810 22:58:54.709007 19837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19fe7586a2cd45ae83c118cfb2afe5a9" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c93871cf3e04ce2b9febf227378ed43" is_pre_election: true
I20260810 22:58:54.709164 19837 raft_consensus.cc:2454] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 0.
I20260810 22:58:54.709163 19610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19fe7586a2cd45ae83c118cfb2afe5a9" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" is_pre_election: true
I20260810 22:58:54.709301 19610 raft_consensus.cc:2454] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 0.
I20260810 22:58:54.709566 19227 leader_election.cc:302] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c93871cf3e04ce2b9febf227378ed43, c5a8217a27684e27aea8fd642e533813; no voters:
I20260810 22:58:54.709694 19996 raft_consensus.cc:2781] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:54.709753 19996 raft_consensus.cc:482] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:54.709776 19996 raft_consensus.cc:3037] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:54.710469 19996 raft_consensus.cc:504] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.710603 19996 leader_election.cc:288] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 election: Requested vote from peers f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555), 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759)
I20260810 22:58:54.710769 19610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19fe7586a2cd45ae83c118cfb2afe5a9" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
I20260810 22:58:54.710845 19610 raft_consensus.cc:3037] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:54.710796 19837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19fe7586a2cd45ae83c118cfb2afe5a9" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c93871cf3e04ce2b9febf227378ed43"
I20260810 22:58:54.710902 19837 raft_consensus.cc:3037] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:54.711583 19610 raft_consensus.cc:2454] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 1.
I20260810 22:58:54.711722 19837 raft_consensus.cc:2454] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 1.
I20260810 22:58:54.711763 19229 leader_election.cc:302] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5a8217a27684e27aea8fd642e533813, f4a6fde08b5b4fda9389ee1c3d3a76ae; no voters:
I20260810 22:58:54.711918 19996 raft_consensus.cc:2781] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:54.712060 19996 raft_consensus.cc:686] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 1 LEADER]: Becoming Leader. State: Replica: c5a8217a27684e27aea8fd642e533813, State: Running, Role: LEADER
I20260810 22:58:54.712208 19996 consensus_queue.cc:237] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } }
I20260810 22:58:54.713240 17612 catalog_manager.cc:5495] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5a8217a27684e27aea8fd642e533813 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "c5a8217a27684e27aea8fd642e533813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:54.753718 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.751459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55772 (local address 127.6.47.129:34731)
0810 22:58:54.751510 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.751512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.751520 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:54.751595 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.751596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.751625 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.751680 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:54.751797 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.752257 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.752969 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.753158 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.753213 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.753357 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:54.753460 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:54.753550 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.753568 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.753595 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:58:54.755635 19837 raft_consensus.cc:1261] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Refusing update from remote peer c5a8217a27684e27aea8fd642e533813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:54.755663 19610 raft_consensus.cc:1261] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Refusing update from remote peer c5a8217a27684e27aea8fd642e533813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:54.756033 20004 consensus_queue.cc:1035] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:54.756189 19996 consensus_queue.cc:1035] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:54.759366 20013 mvcc.cc:205] Tried to move back new op lower bound from 7317105601556180992 to 7317105601381879808. Current Snapshot: MvccSnapshot[applied={T|T < 7317105601556180992}]
I20260810 22:58:54.761217 17612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:54.761318 17612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:54.763116 17612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b28120137c2490091fa71c46ecfc396: Sending DeleteTablet for 3 replicas of tablet 19fe7586a2cd45ae83c118cfb2afe5a9
I20260810 22:58:54.763481 19586 tablet_service.cc:1514] Processing DeleteTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:54 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:34546
I20260810 22:58:54.763844 19327 tablet_service.cc:1514] Processing DeleteTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:54 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:55766
I20260810 22:58:54.764134 20029 tablet_replica.cc:330] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: stopping tablet replica
I20260810 22:58:54.764394 19808 tablet_service.cc:1514] Processing DeleteTablet for tablet 19fe7586a2cd45ae83c118cfb2afe5a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:54 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260810 22:58:54.764433 20029 raft_consensus.cc:2229] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:54.764613 20029 raft_consensus.cc:2258] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:54.764707 20030 tablet_replica.cc:330] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: stopping tablet replica
I20260810 22:58:54.764853 20030 raft_consensus.cc:2229] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:54.764999 20030 raft_consensus.cc:2258] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:54.765760 20029 ts_tablet_manager.cc:1905] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:54.765799 20026 tablet_replica.cc:330] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: stopping tablet replica
I20260810 22:58:54.765821 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 17551
I20260810 22:58:54.774422 20029 ts_tablet_manager.cc:1918] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:54.774508 20029 log.cc:1196] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/19fe7586a2cd45ae83c118cfb2afe5a9
I20260810 22:58:54.774737 20029 ts_tablet_manager.cc:1939] T 19fe7586a2cd45ae83c118cfb2afe5a9 P c5a8217a27684e27aea8fd642e533813: Deleting consensus metadata
I20260810 22:58:54.793128 20030 ts_tablet_manager.cc:1905] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:54.795475 20030 ts_tablet_manager.cc:1918] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:54.795583 20030 log.cc:1196] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/19fe7586a2cd45ae83c118cfb2afe5a9
I20260810 22:58:54.795881 20030 ts_tablet_manager.cc:1939] T 19fe7586a2cd45ae83c118cfb2afe5a9 P 1c93871cf3e04ce2b9febf227378ed43: Deleting consensus metadata
I20260810 22:58:54.765889 20026 raft_consensus.cc:2229] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:54.796418 20026 raft_consensus.cc:2258] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:54.797847 20026 ts_tablet_manager.cc:1905] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:54.799548 20026 ts_tablet_manager.cc:1918] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:54.799683 20026 log.cc:1196] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/19fe7586a2cd45ae83c118cfb2afe5a9
I20260810 22:58:54.799997 20026 ts_tablet_manager.cc:1939] T 19fe7586a2cd45ae83c118cfb2afe5a9 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting consensus metadata
W20260810 22:58:54.800868 19488 connection.cc:535] server connection from 127.0.0.1:34546 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 22:58:54.800875 19739 connection.cc:535] server connection from 127.0.0.1:60774 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 22:58:54.801215 19227 connection.cc:535] server connection from 127.0.0.1:55766 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 22:58:54.805845 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.765157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.6.47.188:41407)
0810 22:58:54.765213 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.765215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.797283 (+ 32068us) server_negotiation.cc:407] Connection header received
0810 22:58:54.797346 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.797348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.797362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.797424 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.798278 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.799068 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.800205 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.800579 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.800621 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.802345 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.802347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.802347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.802354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.804419 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.804817 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.804818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.804818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.804873 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.805043 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.805044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.805044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.805149 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.805239 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.805682 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.805753 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 22:58:54.810953 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.803309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.6.47.188:41407)
0810 22:58:54.803353 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.803355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.803475 (+ 120us) server_negotiation.cc:407] Connection header received
0810 22:58:54.803605 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.803606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.803621 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.803650 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.804389 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.805121 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.806121 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.806373 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.806406 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.807950 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.807951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.807952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.807959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.809781 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.810195 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.810196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.810197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.810256 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.810421 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.810421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.810422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.810561 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.810670 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.810771 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.810845 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 22:58:54.811091 19955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:54.803070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.6.47.189:38843)
0810 22:58:54.803115 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:54.803117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:54.803316 (+ 199us) server_negotiation.cc:407] Connection header received
0810 22:58:54.803424 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:54.803425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:54.803441 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:54.803469 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:54.804543 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.805288 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.806593 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:54.806891 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:54.806968 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:54.808238 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:54.808239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:54.808239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:54.808246 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:54.809816 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.810251 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.810252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.810252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.810308 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:54.810508 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:54.810509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:54.810509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:54.810640 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:54.810727 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:54.810900 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:58:54.810985 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 22:58:55.760638 20043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:55.760306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:43665)
0810 22:58:55.760451 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:55.760580 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
W20260810 22:58:55.760943 19930 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:36187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:55.765304 20046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:55.765054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:40563)
0810 22:58:55.765181 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:55.765264 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":50,"threads_started":1}
I20260810 22:58:55.765480 20049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:55.765267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:53513)
0810 22:58:55.765366 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:55.765438 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
W20260810 22:58:55.765508 19439 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:36187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
W20260810 22:58:55.765669 19694 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:36187 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.263818 20050 raft_consensus.cc:482] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b28120137c2490091fa71c46ecfc396)
I20260810 22:58:56.263952 20050 raft_consensus.cc:504] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } }
I20260810 22:58:56.264241 20050 leader_election.cc:288] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187), cba40bc5ebbe479799858da66d24f307 (127.6.47.189:38843)
I20260810 22:58:56.264674 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.264362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47508)
0810 22:58:56.264557 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.264617 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":48,"threads_started":1}
W20260810 22:58:56.264837 19078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.264977 20056 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8b28120137c2490091fa71c46ecfc396)
I20260810 22:58:56.265151 20056 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } }
I20260810 22:58:56.265317 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.265202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47520)
0810 22:58:56.265259 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.265292 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 22:58:56.265432 20056 leader_election.cc:288] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187), f72ed3a6d87f412883aedbc97c3b6a9a (127.6.47.188:41407)
W20260810 22:58:56.265460 19078 leader_election.cc:334] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187): Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.265690 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.265491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47524)
0810 22:58:56.265590 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.265647 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
W20260810 22:58:56.265844 18730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.266311 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.266192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47526)
0810 22:58:56.266250 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.266280 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:58:56.266412 18730 leader_election.cc:334] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187): Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.267866 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.264438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:38843 (local address 127.0.0.1:53156)
0810 22:58:56.264540 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.264553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.264655 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.264953 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.264955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.264962 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.265021 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.265024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.266034 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.266036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.267057 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.267061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.267605 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.267607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.267675 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.267680 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.267746 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.267801 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":75,"threads_started":1}
I20260810 22:58:56.267879 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.264548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.6.47.189:38843)
0810 22:58:56.264688 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.264690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.264701 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:56.264733 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.264734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.264745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.264811 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.265141 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.265926 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.267152 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.267521 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.267627 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.267640 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.267750 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.267820 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260810 22:58:56.268244 18755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cba40bc5ebbe479799858da66d24f307" is_pre_election: true
I20260810 22:58:56.268352 18755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f72ed3a6d87f412883aedbc97c3b6a9a in term 1.
I20260810 22:58:56.268517 19079 leader_election.cc:302] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cba40bc5ebbe479799858da66d24f307, f72ed3a6d87f412883aedbc97c3b6a9a; no voters: 8b28120137c2490091fa71c46ecfc396
I20260810 22:58:56.268646 20050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 22:58:56.268690 20050 raft_consensus.cc:482] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8b28120137c2490091fa71c46ecfc396)
I20260810 22:58:56.268711 20050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:58:56.269117 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.265729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:41407 (local address 127.0.0.1:58810)
0810 22:58:56.265949 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.265960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.266015 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.266276 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.266277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.266279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.266316 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.266318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.267033 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.267035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.268165 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.268167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.268898 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.268899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.268970 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.268974 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.269039 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.269069 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":2,"thread_start_us":44,"threads_started":1}
I20260810 22:58:56.269156 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.265943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.6.47.188:41407)
0810 22:58:56.266133 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.266135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.266142 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:56.266164 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.266164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.266174 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.266269 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.266448 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.266953 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.268245 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.268787 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.268880 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.268892 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.269065 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.269105 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260810 22:58:56.269507 19129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cba40bc5ebbe479799858da66d24f307" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" is_pre_election: true
I20260810 22:58:56.269790 20050 raft_consensus.cc:504] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } }
I20260810 22:58:56.269939 19129 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cba40bc5ebbe479799858da66d24f307 in current term 2: Already voted for candidate f72ed3a6d87f412883aedbc97c3b6a9a in this term.
I20260810 22:58:56.269968 20050 leader_election.cc:288] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 election: Requested vote from peers 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187), cba40bc5ebbe479799858da66d24f307 (127.6.47.189:38843)
I20260810 22:58:56.270123 18733 leader_election.cc:302] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cba40bc5ebbe479799858da66d24f307; no voters: 8b28120137c2490091fa71c46ecfc396, f72ed3a6d87f412883aedbc97c3b6a9a
I20260810 22:58:56.270159 18755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cba40bc5ebbe479799858da66d24f307"
I20260810 22:58:56.270217 20056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:58:56.270330 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.270035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47536)
0810 22:58:56.270264 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.270299 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260810 22:58:56.270650 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.270549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47542)
0810 22:58:56.270591 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.270630 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 22:58:56.270728 19078 leader_election.cc:334] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187): Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
I20260810 22:58:56.271080 20056 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 22:58:56.271739 18755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f72ed3a6d87f412883aedbc97c3b6a9a in term 2.
I20260810 22:58:56.272176 19079 leader_election.cc:302] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cba40bc5ebbe479799858da66d24f307, f72ed3a6d87f412883aedbc97c3b6a9a; no voters: 8b28120137c2490091fa71c46ecfc396
I20260810 22:58:56.272269 20050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:58:56.272405 20050 raft_consensus.cc:686] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [term 2 LEADER]: Becoming Leader. State: Replica: f72ed3a6d87f412883aedbc97c3b6a9a, State: Running, Role: LEADER
I20260810 22:58:56.272464 20050 consensus_queue.cc:237] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } }
I20260810 22:58:56.272866 20066 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f72ed3a6d87f412883aedbc97c3b6a9a. Latest consensus state: current_term: 2 leader_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:56.272939 20066 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:56.273090 20068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:56.273587 20068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:56.273696 20068 catalog_manager.cc:1259] Loaded cluster ID: 4f2c2286fd894f7b89178f1c114c0b7c
I20260810 22:58:56.273729 20068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:56.273885 20068 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:56.273979 20068 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Loaded TSK: 0
I20260810 22:58:56.274127 20068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:58:56.287586 19104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:56.289348 19104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:56.290344 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.290208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47566)
0810 22:58:56.290262 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.290314 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 22:58:56.290373 18755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [term 2 FOLLOWER]: Refusing update from remote peer f72ed3a6d87f412883aedbc97c3b6a9a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 22:58:56.290654 20050 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 22:58:56.290735 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.290620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.0.0.1:47580)
0810 22:58:56.290677 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.290713 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 22:58:56.290846 19078 consensus_peers.cc:487] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a -> Peer 8b28120137c2490091fa71c46ecfc396 (127.6.47.190:36187): Couldn't send request to peer 8b28120137c2490091fa71c46ecfc396. Status: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 22:58:56.291342 20056 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f72ed3a6d87f412883aedbc97c3b6a9a. Latest consensus state: current_term: 2 leader_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:56.291432 20056 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:58:56.291936 20050 sys_catalog.cc:455] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:56.292026 20050 sys_catalog.cc:458] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:56.292179 20056 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "8b28120137c2490091fa71c46ecfc396" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36187 } } peers { permanent_uuid: "cba40bc5ebbe479799858da66d24f307" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 38843 } attrs { promote: false } } peers { permanent_uuid: "f72ed3a6d87f412883aedbc97c3b6a9a" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 41407 } attrs { promote: false } } }
I20260810 22:58:56.292279 20056 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba40bc5ebbe479799858da66d24f307 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 22:58:56.297793 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34731 (local address 127.0.0.1:55778)
0810 22:58:56.294418 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.294430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.294485 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.294976 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.294977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.294983 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.295058 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.295060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.295992 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.295993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.296885 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.296888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.297578 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.297579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.297636 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.297641 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.297692 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.297732 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":6}
I20260810 22:58:56.298063 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:41759 (local address 127.0.0.1:60782)
0810 22:58:56.294319 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.294330 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.294374 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.295208 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.295208 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.295209 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.295234 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.295235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.296087 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.296088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.297149 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.297152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.297870 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.297872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.297932 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.297936 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.297963 (+ 27us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.298001 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260810 22:58:56.298090 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.6.47.129:34731)
0810 22:58:56.294828 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.294831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.294841 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:56.294870 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.294871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.294883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.294957 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.295141 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.295918 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.296958 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.297496 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.297563 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.297576 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.297970 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.298017 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":68,"threads_started":1}
I20260810 22:58:56.298301 19327 tablet_service.cc:1467] Processing CreateTablet for tablet e16e83a6be844d418c92635932c21070 (DEFAULT_TABLE table=test-table [id=444ee08ff3324075a950b8470bdb549c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:56.298395 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60782 (local address 127.6.47.131:41759)
0810 22:58:56.295031 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.295033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.295040 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:56.295083 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.295084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.295091 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.295203 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.295285 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.295994 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.297242 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.297782 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.298234 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.298249 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.298268 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.298314 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":68,"threads_started":1}
I20260810 22:58:56.298441 19327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e16e83a6be844d418c92635932c21070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:56.298574 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:45555 (local address 127.0.0.1:34560)
0810 22:58:56.294555 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.294566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.294613 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.294977 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.294977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.294980 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.295058 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.295060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.296903 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.296904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.297595 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.297597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.298334 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.298335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.298391 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.298396 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.298434 (+ 38us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.298494 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":22,"thread_start_us":76,"threads_started":1}
I20260810 22:58:56.298637 19808 tablet_service.cc:1467] Processing CreateTablet for tablet e16e83a6be844d418c92635932c21070 (DEFAULT_TABLE table=test-table [id=444ee08ff3324075a950b8470bdb549c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:56.298779 19808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e16e83a6be844d418c92635932c21070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:56.299015 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.294684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34560 (local address 127.6.47.130:45555)
0810 22:58:56.294838 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.294840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.294849 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:56.294874 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.294874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.294886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.294957 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.296026 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.296823 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.297679 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.298245 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.298734 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.298750 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.298770 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.298814 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260810 22:58:56.299194 19586 tablet_service.cc:1467] Processing CreateTablet for tablet e16e83a6be844d418c92635932c21070 (DEFAULT_TABLE table=test-table [id=444ee08ff3324075a950b8470bdb549c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:56.299299 19586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e16e83a6be844d418c92635932c21070. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:56.299925 20098 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813: Bootstrap starting.
I20260810 22:58:56.300208 20099 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Bootstrap starting.
I20260810 22:58:56.300426 20098 tablet_bootstrap.cc:654] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:56.300570 20099 tablet_bootstrap.cc:654] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:56.300844 20101 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Bootstrap starting.
I20260810 22:58:56.301110 20098 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813: No bootstrap required, opened a new log
I20260810 22:58:56.301193 20098 ts_tablet_manager.cc:1397] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:56.301198 20101 tablet_bootstrap.cc:654] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:56.301323 20098 raft_consensus.cc:348] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.301420 20098 raft_consensus.cc:374] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:56.301430 20099 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: No bootstrap required, opened a new log
I20260810 22:58:56.301450 20098 raft_consensus.cc:729] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a8217a27684e27aea8fd642e533813, State: Initialized, Role: FOLLOWER
I20260810 22:58:56.301477 20099 ts_tablet_manager.cc:1397] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:56.301512 20098 consensus_queue.cc:260] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.301606 20099 raft_consensus.cc:348] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.301676 20098 ts_tablet_manager.cc:1428] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:56.301689 20099 raft_consensus.cc:374] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:56.301717 20099 raft_consensus.cc:729] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c93871cf3e04ce2b9febf227378ed43, State: Initialized, Role: FOLLOWER
I20260810 22:58:56.301780 20099 consensus_queue.cc:260] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.302222 20101 tablet_bootstrap.cc:492] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: No bootstrap required, opened a new log
I20260810 22:58:56.302323 20101 ts_tablet_manager.cc:1397] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:56.302436 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.302026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:58899)
0810 22:58:56.302303 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.302341 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260810 22:58:56.302507 19438 heartbeater.cc:503] Master 127.6.47.188:41407 was elected leader, sending a full tablet report...
I20260810 22:58:56.302479 20101 raft_consensus.cc:348] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.302542 20101 raft_consensus.cc:374] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:56.302562 20101 raft_consensus.cc:729] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4a6fde08b5b4fda9389ee1c3d3a76ae, State: Initialized, Role: FOLLOWER
I20260810 22:58:56.302621 20101 consensus_queue.cc:260] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.302804 20101 ts_tablet_manager.cc:1428] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:56.303184 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.302793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:44119)
0810 22:58:56.303109 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.303140 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":85,"threads_started":1}
I20260810 22:58:56.303419 19692 heartbeater.cc:503] Master 127.6.47.188:41407 was elected leader, sending a full tablet report...
I20260810 22:58:56.303897 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.303697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:50791)
0810 22:58:56.303819 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.303855 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":80,"threads_started":1}
I20260810 22:58:56.304149 20099 ts_tablet_manager.cc:1428] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260810 22:58:56.304221 19928 heartbeater.cc:503] Master 127.6.47.188:41407 was elected leader, sending a full tablet report...
I20260810 22:58:56.304598 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.304339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:57395)
0810 22:58:56.304375 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.304566 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 22:58:56.304612 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.303075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:44733)
0810 22:58:56.304407 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.304566 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1307,"thread_start_us":82,"threads_started":1}
I20260810 22:58:56.304913 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.304837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:38427)
0810 22:58:56.304870 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.304894 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260810 22:58:56.434356 20116 raft_consensus.cc:482] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:56.434463 20116 raft_consensus.cc:504] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.434674 20107 raft_consensus.cc:482] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:56.434756 20107 raft_consensus.cc:504] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.434903 20107 leader_election.cc:288] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759), f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:56.434904 20116 leader_election.cc:288] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731), f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:56.435148 19610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" is_pre_election: true
I20260810 22:58:56.435256 19610 raft_consensus.cc:2454] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 0.
I20260810 22:58:56.435418 19837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c93871cf3e04ce2b9febf227378ed43" is_pre_election: true
I20260810 22:58:56.435434 19229 leader_election.cc:302] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5a8217a27684e27aea8fd642e533813, f4a6fde08b5b4fda9389ee1c3d3a76ae; no voters:
I20260810 22:58:56.435498 19837 raft_consensus.cc:2454] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 0.
I20260810 22:58:56.435592 20107 raft_consensus.cc:2781] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:56.435655 20107 raft_consensus.cc:482] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:56.435678 20107 raft_consensus.cc:3037] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:56.436708 20107 raft_consensus.cc:504] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.436843 20107 leader_election.cc:288] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 election: Requested vote from peers 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759), f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555)
I20260810 22:58:56.437049 19837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c93871cf3e04ce2b9febf227378ed43"
I20260810 22:58:56.437072 19610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "c5a8217a27684e27aea8fd642e533813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae"
I20260810 22:58:56.437121 19837 raft_consensus.cc:3037] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:56.437151 19610 raft_consensus.cc:3037] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:56.437835 19837 raft_consensus.cc:2454] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 1.
I20260810 22:58:56.437981 19610 raft_consensus.cc:2454] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5a8217a27684e27aea8fd642e533813 in term 1.
I20260810 22:58:56.437983 19227 leader_election.cc:302] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c93871cf3e04ce2b9febf227378ed43, c5a8217a27684e27aea8fd642e533813; no voters:
I20260810 22:58:56.438054 20107 raft_consensus.cc:2781] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:56.438169 20107 raft_consensus.cc:686] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [term 1 LEADER]: Becoming Leader. State: Replica: c5a8217a27684e27aea8fd642e533813, State: Running, Role: LEADER
I20260810 22:58:56.438231 20107 consensus_queue.cc:237] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } }
I20260810 22:58:56.438724 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.438609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:58837)
0810 22:58:56.438659 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.438693 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 22:58:56.438993 19104 catalog_manager.cc:5495] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5a8217a27684e27aea8fd642e533813 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "c5a8217a27684e27aea8fd642e533813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5a8217a27684e27aea8fd642e533813" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:56.439256 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.435315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:56235 (local address 127.6.47.130:45555)
0810 22:58:56.435358 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.435360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.435484 (+ 124us) server_negotiation.cc:407] Connection header received
0810 22:58:56.436026 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.436028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.436042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.436111 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.436250 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.436942 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.438282 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.438787 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.438834 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.438843 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.439078 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.439135 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 22:58:56.439504 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.435208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:45555 (local address 127.6.47.131:56235)
0810 22:58:56.435418 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.435428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.435623 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.436126 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.436127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.436129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.436171 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.436172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.437047 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.437048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.438163 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.438166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.438919 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.438920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.438990 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.438993 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.439384 (+ 391us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.439432 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":104,"threads_started":1}
I20260810 22:58:56.439517 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.435060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34731 (local address 127.6.47.131:57185)
0810 22:58:56.435153 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.435167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:56.435223 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:56.435596 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:56.435598 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:56.435600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:56.435679 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.435681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.436586 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.436587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.437696 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:56.437700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.438466 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.438467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:56.438544 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.438547 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:56.439429 (+ 882us) client_negotiation.cc:240] Negotiation successful
0810 22:58:56.439461 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 22:58:56.439788 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.435397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:57185 (local address 127.6.47.129:34731)
0810 22:58:56.435446 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.435448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.435456 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:58:56.435480 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.435481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.435494 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.435541 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:56.435800 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.436484 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.437799 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.438351 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.438396 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.438406 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.438625 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.438683 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 22:58:56.439782 19610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "1c93871cf3e04ce2b9febf227378ed43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" is_pre_election: true
I20260810 22:58:56.439884 19610 raft_consensus.cc:2379] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1c93871cf3e04ce2b9febf227378ed43 in current term 1: Already voted for candidate c5a8217a27684e27aea8fd642e533813 in this term.
I20260810 22:58:56.439965 19362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e16e83a6be844d418c92635932c21070" candidate_uuid: "1c93871cf3e04ce2b9febf227378ed43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5a8217a27684e27aea8fd642e533813" is_pre_election: true
I20260810 22:58:56.440299 19739 leader_election.cc:302] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1c93871cf3e04ce2b9febf227378ed43; no voters: c5a8217a27684e27aea8fd642e533813, f4a6fde08b5b4fda9389ee1c3d3a76ae
I20260810 22:58:56.440472 20116 raft_consensus.cc:2726] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 22:58:56.480334 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.478045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.6.47.129:34731)
0810 22:58:56.478111 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:56.478114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:56.478124 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:58:56.478152 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:56.478152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:56.478163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:56.478207 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:56.478348 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.478861 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.479617 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:56.479823 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:56.479902 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:56.480045 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:56.480085 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:56.480145 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:56.480217 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 22:58:56.480243 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:58:56.481341 19837 raft_consensus.cc:1261] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Refusing update from remote peer c5a8217a27684e27aea8fd642e533813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:56.481343 19610 raft_consensus.cc:1261] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Refusing update from remote peer c5a8217a27684e27aea8fd642e533813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:56.481626 20120 consensus_queue.cc:1035] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c93871cf3e04ce2b9febf227378ed43" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:56.481750 20107 consensus_queue.cc:1035] T e16e83a6be844d418c92635932c21070 P c5a8217a27684e27aea8fd642e533813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4a6fde08b5b4fda9389ee1c3d3a76ae" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 45555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:56.482699 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.482597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:48885)
0810 22:58:56.482652 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.482673 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 22:58:56.482690 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.482503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:45709)
0810 22:58:56.482608 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.482642 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 22:58:56.483024 20127 mvcc.cc:205] Tried to move back new op lower bound from 7317105608625549312 to 7317105608451424256. Current Snapshot: MvccSnapshot[applied={T|T < 7317105608625549312}]
I20260810 22:58:56.483717 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.483577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:32801)
0810 22:58:56.483638 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.483690 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 22:58:56.484895 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.484805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.130:50839)
0810 22:58:56.484840 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.484865 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 22:58:56.485070 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.484688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.129:53333)
0810 22:58:56.485012 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.485041 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307}
I20260810 22:58:56.485175 19101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:56.485294 19101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:56.485450 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:56.485361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:36187 (local address 127.6.47.131:46951)
0810 22:58:56.485397 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.485425 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:36187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 22:58:56.486732 19101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f72ed3a6d87f412883aedbc97c3b6a9a: Sending DeleteTablet for 3 replicas of tablet e16e83a6be844d418c92635932c21070
I20260810 22:58:56.486987 19808 tablet_service.cc:1514] Processing DeleteTablet for tablet e16e83a6be844d418c92635932c21070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:56 UTC) from {username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:60782
I20260810 22:58:56.487061 19327 tablet_service.cc:1514] Processing DeleteTablet for tablet e16e83a6be844d418c92635932c21070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:56 UTC) from {username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:55778
I20260810 22:58:56.487183 19586 tablet_service.cc:1514] Processing DeleteTablet for tablet e16e83a6be844d418c92635932c21070 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:56 UTC) from {username='oryx', principal='oryx/127.6.47.188@KRBTEST.COM'} at 127.0.0.1:34560
I20260810 22:58:56.487214 20146 tablet_replica.cc:330] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: stopping tablet replica
I20260810 22:58:56.487305 20146 raft_consensus.cc:2229] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:56.487371 20146 raft_consensus.cc:2258] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:56.487376 20149 tablet_replica.cc:330] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: stopping tablet replica
I20260810 22:58:56.487442 20149 raft_consensus.cc:2229] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:56.487450 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 19027
I20260810 22:58:56.487490 20149 raft_consensus.cc:2258] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:56.490923 20149 ts_tablet_manager.cc:1905] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:56.493199 20149 ts_tablet_manager.cc:1918] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:56.493275 20149 log.cc:1196] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/e16e83a6be844d418c92635932c21070
I20260810 22:58:56.493549 20149 ts_tablet_manager.cc:1939] T e16e83a6be844d418c92635932c21070 P f4a6fde08b5b4fda9389ee1c3d3a76ae: Deleting consensus metadata
I20260810 22:58:56.493921 19080 catalog_manager.cc:4841] TS f4a6fde08b5b4fda9389ee1c3d3a76ae (127.6.47.130:45555): tablet e16e83a6be844d418c92635932c21070 (table test-table [id=444ee08ff3324075a950b8470bdb549c]) successfully deleted
I20260810 22:58:56.499858 20146 ts_tablet_manager.cc:1905] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:56.502526 20146 ts_tablet_manager.cc:1918] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:56.502652 20146 log.cc:1196] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/e16e83a6be844d418c92635932c21070
I20260810 22:58:56.502926 20146 ts_tablet_manager.cc:1939] T e16e83a6be844d418c92635932c21070 P 1c93871cf3e04ce2b9febf227378ed43: Deleting consensus metadata
I20260810 22:58:56.503340 19079 catalog_manager.cc:4841] TS 1c93871cf3e04ce2b9febf227378ed43 (127.6.47.131:41759): tablet e16e83a6be844d418c92635932c21070 (table test-table [id=444ee08ff3324075a950b8470bdb549c]) successfully deleted
W20260810 22:58:56.508945 20083 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:56.508780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34731 (local address 127.0.0.1:55786)
0810 22:58:56.508844 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:56.508875 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:34731: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 22:58:56.509094 19078 catalog_manager.cc:4584] TS c5a8217a27684e27aea8fd642e533813 (127.6.47.129:34731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e16e83a6be844d418c92635932c21070: Network error: Client connection negotiation failed: client connection to 127.6.47.129:34731: connect: Connection reset by peer (error 104)
I20260810 22:58:56.509130 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 19448
I20260810 22:58:56.530623 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 19706
I20260810 22:58:56.543210 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 18510
I20260810 22:58:56.552245 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 18846
2026-08-10T22:58:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5185 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:58:56 dist-test-slave-rwrg krb5kdc[20155](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:58:56 dist-test-slave-rwrg krb5kdc[20155](info): set up 2 sockets
Aug 10 22:58:56 dist-test-slave-rwrg krb5kdc[20155](info): commencing operation
krb5kdc: starting...
W20260810 22:58:57.838225 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:58:57 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:58:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:58:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab.
Entry for principal oryx/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab.
I20260810 22:58:57.959100 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:45251
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab
--principal=oryx/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:45251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:58.041368 20174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:58.041646 20174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:58.041685 20174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:58.042862 20174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:58:58.042934 20174 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 22:58:58.042956 20174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:58.042979 20174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:58.043008 20174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:58:58.043040 20174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:58:58.044660 20174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:45251
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.190.keytab
--principal=oryx/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:58.044965 20174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:58.045233 20174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:58.050316 20180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.050360 20183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.050529 20174 server_base.cc:1034] running on GCE node
W20260810 22:58:58.050333 20189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.050843 20174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:58.051205 20174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:58.052390 20174 hybrid_clock.cc:648] HybridClock initialized: now 1786402738052367 us; error 41 us; skew 500 ppm
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:58.055073 20174 init.cc:377] Logged in from keytab as oryx/127.6.47.190@KRBTEST.COM (short username oryx)
I20260810 22:58:58.055532 20174 webserver.cc:460] Webserver started at http://127.6.47.190:41187/ using document root <none> and password file <none>
I20260810 22:58:58.055799 20174 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:58.055872 20174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:58.056030 20174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:58.057240 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "ca4150231bf940ee9d2790e19c96def5"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "5e123b59c7c3bdd3d5ab208069752198"
server_key_iv: "278b761d13adadf76c5b003f63343088"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.057652 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "ca4150231bf940ee9d2790e19c96def5"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "5e123b59c7c3bdd3d5ab208069752198"
server_key_iv: "278b761d13adadf76c5b003f63343088"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.059113 20174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:58.059850 20197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.060029 20174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:58.060148 20174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "ca4150231bf940ee9d2790e19c96def5"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "5e123b59c7c3bdd3d5ab208069752198"
server_key_iv: "278b761d13adadf76c5b003f63343088"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.060226 20174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:58.073030 20174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:58.075409 20174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:58.075583 20174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:58.080538 20174 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:45251
I20260810 22:58:58.080619 20304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:45251 every 8 connection(s)
I20260810 22:58:58.081007 20174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:58:58.081743 20305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.083841 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 20174
I20260810 22:58:58.083923 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:58:58.084031 20305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Bootstrap starting.
I20260810 22:58:58.084138 6334 external_mini_cluster.cc:1428] Setting key 74381173ede997f9ff810aaa435f0bb2
I20260810 22:58:58.084884 20305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.085240 20305 log.cc:824] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:58.086006 20305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: No bootstrap required, opened a new log
I20260810 22:58:58.088353 20305 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } }
I20260810 22:58:58.088574 20305 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.088629 20305 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4150231bf940ee9d2790e19c96def5, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.088786 20305 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } }
I20260810 22:58:58.088861 20305 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:58:58.088891 20305 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:58:58.088934 20305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:58.089689 20305 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } }
I20260810 22:58:58.089808 20305 leader_election.cc:302] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca4150231bf940ee9d2790e19c96def5; no voters:
I20260810 22:58:58.090003 20305 leader_election.cc:288] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:58:58.090029 20312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:58.090161 20312 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [term 1 LEADER]: Becoming Leader. State: Replica: ca4150231bf940ee9d2790e19c96def5, State: Running, Role: LEADER
I20260810 22:58:58.090265 20312 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } }
I20260810 22:58:58.090298 20305 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:58:58.090638 20315 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca4150231bf940ee9d2790e19c96def5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } } }
I20260810 22:58:58.090896 20315 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [sys.catalog]: This master's current role is: LEADER
I20260810 22:58:58.090821 20314 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca4150231bf940ee9d2790e19c96def5. Latest consensus state: current_term: 1 leader_uuid: "ca4150231bf940ee9d2790e19c96def5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4150231bf940ee9d2790e19c96def5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 45251 } } }
I20260810 22:58:58.091197 20329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:58:58.091202 20314 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5 [sys.catalog]: This master's current role is: LEADER
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:58.091737 20329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:58:58.093796 20329 catalog_manager.cc:1347] Generated new cluster ID: 5f0e07763bbf43c39c6485a3ea493fe2
I20260810 22:58:58.093851 20329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:58:58.095731 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.084694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.6.47.190:45251)
0810 22:58:58.084884 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.084893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.084952 (+ 59us) server_negotiation.cc:407] Connection header received
0810 22:58:58.085530 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.085538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.085585 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.085758 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.086412 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.087848 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.088622 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.089272 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.089391 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.092324 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.092328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.092329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.092340 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.094237 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.094602 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.094605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.094605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.094702 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.094919 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.094920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.094920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.095154 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.095306 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.095424 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.095557 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260810 22:58:58.121915 20329 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:58:58.122520 20329 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:58:58.125349 20329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Generated new TSK 0
I20260810 22:58:58.125530 20329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
I20260810 22:58:58.159312 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45251
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:58.243093 20348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:58.243346 20348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:58.243389 20348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:58.244552 20348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:58.244624 20348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:58.244660 20348 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:58.244710 20348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:58:58.246174 20348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:58.246434 20348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:58.246739 20348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:58.252028 20357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.252079 20356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.252028 20359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.252308 20348 server_base.cc:1034] running on GCE node
I20260810 22:58:58.252589 20348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:58.252859 20348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:58.254045 20348 hybrid_clock.cc:648] HybridClock initialized: now 1786402738254012 us; error 43 us; skew 500 ppm
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:58.256641 20348 init.cc:377] Logged in from keytab as oryx/127.6.47.129@KRBTEST.COM (short username oryx)
I20260810 22:58:58.257077 20348 webserver.cc:460] Webserver started at http://127.6.47.129:42147/ using document root <none> and password file <none>
I20260810 22:58:58.257257 20348 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:58.257299 20348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:58.257393 20348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:58.258410 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "4a0c3468e0cb42cbb8216476bbb080df"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "c592de89204fd4977ba7c573b56a6e24"
server_key_iv: "4e788f2392c832c0e7e40704f80d1662"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.258716 20348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "4a0c3468e0cb42cbb8216476bbb080df"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "c592de89204fd4977ba7c573b56a6e24"
server_key_iv: "4e788f2392c832c0e7e40704f80d1662"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.259985 20348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:58.260703 20374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.260857 20348 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:58.260960 20348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "4a0c3468e0cb42cbb8216476bbb080df"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "c592de89204fd4977ba7c573b56a6e24"
server_key_iv: "4e788f2392c832c0e7e40704f80d1662"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.261013 20348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:58.278560 20348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:58.280929 20348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:58.281044 20348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:58.281533 20348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:58.281571 20348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.281594 20348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:58.281647 20348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.288988 20348 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:45207
I20260810 22:58:58.289080 20583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:45207 every 8 connection(s)
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:58.289431 20348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:58:58.292986 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.282121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:39829 (local address 127.6.47.190:45251)
0810 22:58:58.282193 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.282195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.282383 (+ 188us) server_negotiation.cc:407] Connection header received
0810 22:58:58.282908 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.282909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.282922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.283000 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.283961 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.284853 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.286556 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.286951 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.287041 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.289769 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.289771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.289772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.289780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.291444 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.291918 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.291920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.291920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.291998 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.292300 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.292302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.292303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.292426 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.292525 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.292810 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.292897 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 22:58:58.293165 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.282072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.129:39829)
0810 22:58:58.282262 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.282304 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.282795 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.283124 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.283131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.283529 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.283850 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.283857 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.284955 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.284956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.286447 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.286451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.287060 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.287061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.287197 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.287782 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.287793 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.289676 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.291547 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.291554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.291557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.291789 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.292097 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.292098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.292098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.292214 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.292520 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.292529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.292672 (+ 143us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.292785 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.292940 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260810 22:58:58.293712 20584 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45251
I20260810 22:58:58.293787 20584 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:58.293977 20584 heartbeater.cc:511] Master 127.6.47.190:45251 requested a full tablet report, sending...
I20260810 22:58:58.294625 20226 ts_manager.cc:194] Registered new tserver with Master: 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207)
I20260810 22:58:58.295588 20226 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:39829
I20260810 22:58:58.295924 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 20348
I20260810 22:58:58.296069 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:58:58.296284 6334 external_mini_cluster.cc:1428] Setting key efb8f4a30a65febd518def599f40440e
I20260810 22:58:58.300539 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.294788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:52731 (local address 127.6.47.190:45251)
0810 22:58:58.294829 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.294831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.294936 (+ 105us) server_negotiation.cc:407] Connection header received
0810 22:58:58.295000 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.295000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.295009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.295056 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.295409 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.295914 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.296954 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.297125 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.297224 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.298096 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.298098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.298098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.298105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.299357 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.299701 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.299703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.299703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.299764 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.299973 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.299974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.299974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.300084 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.300154 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.300351 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.300423 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 22:58:58.300570 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.294710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.129:52731)
0810 22:58:58.294862 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.294874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.294943 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.295062 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.295063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.295275 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.295342 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.295344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.296020 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.296021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.296874 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.296877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.297225 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.297226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.297297 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.297727 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.297731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.298004 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.299445 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.299447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.299447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.299620 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.299832 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.299833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.299833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.299893 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.300159 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.300160 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.300237 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.300309 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.300414 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
I20260810 22:58:58.323737 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45251
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:58.408620 20596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:58.408850 20596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:58.408874 20596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:58.410074 20596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:58.410118 20596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:58.410141 20596 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:58.410158 20596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:58:58.411583 20596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:58.411803 20596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:58.411973 20596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:58.416872 20604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.416880 20605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.417124 20596 server_base.cc:1034] running on GCE node
W20260810 22:58:58.416880 20608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.417483 20596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:58.417747 20596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:58.418910 20596 hybrid_clock.cc:648] HybridClock initialized: now 1786402738418901 us; error 47 us; skew 500 ppm
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:58.421463 20596 init.cc:377] Logged in from keytab as oryx/127.6.47.130@KRBTEST.COM (short username oryx)
I20260810 22:58:58.421875 20596 webserver.cc:460] Webserver started at http://127.6.47.130:33427/ using document root <none> and password file <none>
I20260810 22:58:58.422108 20596 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:58.422194 20596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:58.422363 20596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:58.423395 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "21e38be1700c454d86402e456446a0cf"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "290771629cea01595485204eb00016cb"
server_key_iv: "88fcb97af1ed47586c54c52c19bfd7fc"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.423779 20596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "21e38be1700c454d86402e456446a0cf"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "290771629cea01595485204eb00016cb"
server_key_iv: "88fcb97af1ed47586c54c52c19bfd7fc"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.425158 20596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:58:58.425860 20627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.426025 20596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:58.426133 20596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "21e38be1700c454d86402e456446a0cf"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "290771629cea01595485204eb00016cb"
server_key_iv: "88fcb97af1ed47586c54c52c19bfd7fc"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.426213 20596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:58.444133 20596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:58.446038 20596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:58.446213 20596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:58.446723 20596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:58.446785 20596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.446831 20596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:58.446861 20596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.454475 20596 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:35865
I20260810 22:58:58.454559 20817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:35865 every 8 connection(s)
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:58.454898 20596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:58:58.457726 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.447328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:36451 (local address 127.6.47.190:45251)
0810 22:58:58.447397 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.447399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.447595 (+ 196us) server_negotiation.cc:407] Connection header received
0810 22:58:58.448364 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.448366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.448377 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.448457 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.449477 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.450268 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.451996 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.452312 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.452398 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.455307 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.455309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.455309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.455318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.456585 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.456955 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.456956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.456956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.457007 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.457178 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.457179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.457179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.457270 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.457314 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.457598 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.457660 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 22:58:58.457903 20643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.447236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.130:36451)
0810 22:58:58.447445 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.447495 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.448226 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.448587 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.448592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.449025 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.449335 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.449344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.450384 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.450385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.451883 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.451887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.452435 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.452436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.452621 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.453215 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.453226 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.455200 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.456665 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.456672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.456673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.456887 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.457047 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.457047 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.457048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.457136 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.457339 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.457346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.457478 (+ 132us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.457564 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.457682 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260810 22:58:58.458353 20818 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45251
I20260810 22:58:58.458449 20818 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:58.458642 20818 heartbeater.cc:511] Master 127.6.47.190:45251 requested a full tablet report, sending...
I20260810 22:58:58.459095 20228 ts_manager.cc:194] Registered new tserver with Master: 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:58.459518 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 20596
I20260810 22:58:58.459582 20228 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:36451
I20260810 22:58:58.459631 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:58:58.459851 6334 external_mini_cluster.cc:1428] Setting key 032d5b48b6c02b737eaf0a649a2a3ce1
I20260810 22:58:58.465926 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.459394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.130:50533)
0810 22:58:58.459528 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.459538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.459593 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.459703 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.459703 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.459879 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.459967 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.459970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.460825 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.460827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.461632 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.461635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.462200 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.462201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.462275 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.462831 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.462836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.463122 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.464824 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.464825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.464825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.465045 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.465306 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.465306 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.465306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.465358 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.465692 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.465693 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.465728 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.465778 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.465850 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260810 22:58:58.465981 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.459443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:50533 (local address 127.6.47.190:45251)
0810 22:58:58.459520 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.459522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.459622 (+ 100us) server_negotiation.cc:407] Connection header received
0810 22:58:58.459644 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.459645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.459653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.459701 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.460078 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.460723 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.461780 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.462121 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.462197 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.463233 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.463234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.463235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.463241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.464746 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.465144 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.465145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.465146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.465213 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.465424 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.465425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.465426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.465621 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.465704 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.465806 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.465887 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for oryx/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
I20260810 22:58:58.485641 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:45251
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:58:58.569448 20831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:58:58.569645 20831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:58:58.569669 20831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:58:58.570807 20831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:58:58.570849 20831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:58:58.570869 20831 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 22:58:58.570925 20831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:58:58.572425 20831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.47.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:58:58.572682 20831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:58:58.572923 20831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:58:58.577975 20842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.577986 20843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:58:58.578027 20845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:58:58.578351 20831 server_base.cc:1034] running on GCE node
I20260810 22:58:58.578549 20831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:58:58.578814 20831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:58:58.579954 20831 hybrid_clock.cc:648] HybridClock initialized: now 1786402738579945 us; error 63 us; skew 500 ppm
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:58:58.582530 20831 init.cc:377] Logged in from keytab as oryx/127.6.47.131@KRBTEST.COM (short username oryx)
I20260810 22:58:58.583026 20831 webserver.cc:460] Webserver started at http://127.6.47.131:45191/ using document root <none> and password file <none>
I20260810 22:58:58.583279 20831 fs_manager.cc:359] Metadata directory not provided
I20260810 22:58:58.583348 20831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:58:58.583503 20831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:58:58.584611 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "f69d60b471fd4aae939a09780c588832"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "bd1efcb5f4a03204c126891dfab6ed30"
server_key_iv: "c73ba777da19ba7ce751aa1001a53632"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.584980 20831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "f69d60b471fd4aae939a09780c588832"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "bd1efcb5f4a03204c126891dfab6ed30"
server_key_iv: "c73ba777da19ba7ce751aa1001a53632"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.586320 20831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:58:58.587044 20855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.587204 20831 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:58.587314 20831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "f69d60b471fd4aae939a09780c588832"
format_stamp: "Formatted at 2026-08-10 22:58:58 on dist-test-slave-rwrg"
server_key: "bd1efcb5f4a03204c126891dfab6ed30"
server_key_iv: "c73ba777da19ba7ce751aa1001a53632"
server_key_version: "encryptionkey@0"
I20260810 22:58:58.587389 20831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:58:58.599835 20831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:58:58.601797 20831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:58:58.601974 20831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:58:58.602533 20831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:58:58.602591 20831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.602630 20831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:58:58.602653 20831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:58.610321 20831 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:37403
I20260810 22:58:58.610410 21045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:37403 every 8 connection(s)
I20260810 22:58:58.610735 20831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:58:58.610962 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 20831
I20260810 22:58:58.611074 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:58:58.611299 6334 external_mini_cluster.cc:1428] Setting key 9734d69fde8a182eeb0ca337d09cc71a
I20260810 22:58:58.612375 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.602985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:37067 (local address 127.6.47.190:45251)
0810 22:58:58.603079 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.603082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.603263 (+ 181us) server_negotiation.cc:407] Connection header received
0810 22:58:58.603731 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.603733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.603748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.603828 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.604538 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.605057 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.606097 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.606345 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.606438 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.609128 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.609129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.609130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.609137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.610751 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.611272 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.611274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.611274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.611341 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.611636 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.611637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.611638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.611788 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.611879 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.612179 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.612279 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 22:58:58.612555 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.602966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.131:37067)
0810 22:58:58.603125 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.603166 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.603636 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.603897 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.603901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.604144 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.604393 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.604398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.605164 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.605165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.605980 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.605983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.606459 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.606461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.606586 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.607148 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.607156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.609023 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.610870 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.610878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.610881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.611200 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.611421 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.611422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.611423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.611561 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.611871 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.611878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.612017 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.612141 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.612306 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260810 22:58:58.613013 21046 heartbeater.cc:347] Connected to a master server at 127.6.47.190:45251
I20260810 22:58:58.613099 21046 heartbeater.cc:464] Registering TS with master...
I20260810 22:58:58.613251 21046 heartbeater.cc:511] Master 127.6.47.190:45251 requested a full tablet report, sending...
I20260810 22:58:58.613657 20228 ts_manager.cc:194] Registered new tserver with Master: f69d60b471fd4aae939a09780c588832 (127.6.47.131:37403)
I20260810 22:58:58.614058 20228 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:37067
I20260810 22:58:58.619685 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.614203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:55453 (local address 127.6.47.190:45251)
0810 22:58:58.614242 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.614244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.614301 (+ 57us) server_negotiation.cc:407] Connection header received
0810 22:58:58.614372 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.614373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.614382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.614425 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.614819 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.615289 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.616001 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.616203 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.616286 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.617164 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.617166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.617166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.617173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.618525 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.618908 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.618909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.618909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.618969 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.619166 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.619167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.619168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.619275 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.619344 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.619514 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.619586 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 22:58:58.619693 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.614111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.6.47.131:55453)
0810 22:58:58.614248 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.614259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.614317 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.614438 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.614439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.614674 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:58.614736 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.614738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.615395 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.615397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.615901 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.615903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.616289 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.616290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.616373 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.616796 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:58.616801 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:58.617074 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:58.618625 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.618626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.618627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.618824 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.619037 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:58.619038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:58.619038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:58.619102 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:58.619349 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:58.619350 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:58.619406 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.619488 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.619588 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260810 22:58:58.622962 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:58:58.645501 20310 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:58.639871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.6.47.190:45251)
0810 22:58:58.639931 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.639933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.639944 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:58.640041 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.640043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.640060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.640129 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.640767 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.641242 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.641971 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.642155 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.642232 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.645404 (+ 3172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38494: BlockingRecv error: recv got EOF from 127.0.0.1:38494 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 10 22:58:58 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:58.652714 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.646277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.6.47.190:45251)
0810 22:58:58.646316 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.646318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.646362 (+ 44us) server_negotiation.cc:407] Connection header received
0810 22:58:58.646415 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.646415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.646423 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.646467 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:58.646922 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.647405 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.648050 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.648220 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.648270 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.650327 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:58.650329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:58.650329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:58.650336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:58.651761 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.652047 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.652048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.652048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.652098 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:58.652251 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:58.652251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:58.652251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:58.652346 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:58.652413 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.652551 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.652629 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:58:58.654398 20228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:58.656308 20228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:58.662822 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.658991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.6.47.129:45207)
0810 22:58:58.659183 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.659187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.659247 (+ 60us) server_negotiation.cc:407] Connection header received
0810 22:58:58.659406 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.659409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.659426 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.659715 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:58.659779 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660424 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.661780 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.662383 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.662474 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.662610 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.662690 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.662730 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260810 22:58:58.662810 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.658687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45207 (local address 127.0.0.1:53062)
0810 22:58:58.659183 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.659193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.659240 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.659545 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.659549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.659551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:58.659630 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.659632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660528 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.660529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.661690 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.661693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.662474 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.662475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.662568 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.662584 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.662667 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.662724 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":106,"threads_started":1}
I20260810 22:58:58.663312 21086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.658683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37403 (local address 127.0.0.1:43682)
0810 22:58:58.658970 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.659009 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.659063 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.659631 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.659631 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.659633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:58.659678 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.659680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660913 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.660914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.661927 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.661930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.663099 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.663100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.663162 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.663165 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.663206 (+ 41us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.663238 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":41,"thread_start_us":61,"threads_started":1}
I20260810 22:58:58.663312 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.658684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:35865 (local address 127.0.0.1:51146)
0810 22:58:58.659016 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.659027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.659084 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.659569 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.659570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.659573 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:58.659640 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.659642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660927 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.660928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.661956 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.661958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.663072 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.663073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.663141 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.663145 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.663181 (+ 36us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.663227 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":28,"thread_start_us":109,"threads_started":1}
I20260810 22:58:58.663805 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.658787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.6.47.131:37403)
0810 22:58:58.659493 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.659497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.659506 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:58.659545 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.659547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.659562 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.659635 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:58.659736 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660810 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.662023 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.662775 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.662854 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.663640 (+ 786us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.663670 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.663708 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":115,"threads_started":1}
I20260810 22:58:58.663795 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.659238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.6.47.130:35865)
0810 22:58:58.659385 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.659391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.659402 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:58.659444 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.659446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.659461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.659728 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:58.659791 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.660842 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.662063 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.662785 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.663359 (+ 574us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.663623 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.663655 (+ 32us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.663696 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260810 22:58:58.663687 20468 tablet_service.cc:1467] Processing CreateTablet for tablet ab99ba3abbf4417db82f9c4024992c21 (DEFAULT_TABLE table=test-table [id=37ae73ca04fa493ca260ddff4883c495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.664137 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99ba3abbf4417db82f9c4024992c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.664880 20946 tablet_service.cc:1467] Processing CreateTablet for tablet ab99ba3abbf4417db82f9c4024992c21 (DEFAULT_TABLE table=test-table [id=37ae73ca04fa493ca260ddff4883c495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.665194 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99ba3abbf4417db82f9c4024992c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.667618 21100 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Bootstrap starting.
I20260810 22:58:58.668206 21102 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Bootstrap starting.
I20260810 22:58:58.668457 21100 tablet_bootstrap.cc:654] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.668766 20711 tablet_service.cc:1467] Processing CreateTablet for tablet ab99ba3abbf4417db82f9c4024992c21 (DEFAULT_TABLE table=test-table [id=37ae73ca04fa493ca260ddff4883c495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.668843 21100 log.cc:824] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:58.668912 21102 tablet_bootstrap.cc:654] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.669114 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab99ba3abbf4417db82f9c4024992c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.669304 21102 log.cc:824] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:58.669744 21100 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: No bootstrap required, opened a new log
I20260810 22:58:58.669842 21100 ts_tablet_manager.cc:1397] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 22:58:58.670118 21102 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: No bootstrap required, opened a new log
I20260810 22:58:58.670202 21102 ts_tablet_manager.cc:1397] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:58:58.671918 21100 raft_consensus.cc:348] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.672098 21100 raft_consensus.cc:374] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.672142 21100 raft_consensus.cc:729] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0c3468e0cb42cbb8216476bbb080df, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.672128 21102 raft_consensus.cc:348] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.672293 21102 raft_consensus.cc:374] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.672336 21102 raft_consensus.cc:729] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69d60b471fd4aae939a09780c588832, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.672286 21100 consensus_queue.cc:260] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.672487 21102 consensus_queue.cc:260] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.672714 21100 ts_tablet_manager.cc:1428] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 22:58:58.672791 21102 ts_tablet_manager.cc:1428] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 22:58:58.672860 20584 heartbeater.cc:503] Master 127.6.47.190:45251 was elected leader, sending a full tablet report...
I20260810 22:58:58.672849 21108 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Bootstrap starting.
I20260810 22:58:58.672905 21046 heartbeater.cc:503] Master 127.6.47.190:45251 was elected leader, sending a full tablet report...
I20260810 22:58:58.673525 21108 tablet_bootstrap.cc:654] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.673892 21108 log.cc:824] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Log is configured to *not* fsync() on all Append() calls
I20260810 22:58:58.674594 21108 tablet_bootstrap.cc:492] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: No bootstrap required, opened a new log
I20260810 22:58:58.674695 21108 ts_tablet_manager.cc:1397] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 22:58:58.676014 21108 raft_consensus.cc:348] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.676167 21108 raft_consensus.cc:374] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.676213 21108 raft_consensus.cc:729] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e38be1700c454d86402e456446a0cf, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.676342 21108 consensus_queue.cc:260] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.676725 21108 ts_tablet_manager.cc:1428] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 22:58:58.676746 20818 heartbeater.cc:503] Master 127.6.47.190:45251 was elected leader, sending a full tablet report...
W20260810 22:58:58.705483 20819 tablet.cc:2369] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:58.790226 20585 tablet.cc:2369] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:58:58.861410 21047 tablet.cc:2369] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:58:58.863114 21112 raft_consensus.cc:482] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:58.863206 21112 raft_consensus.cc:504] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.863574 21112 leader_election.cc:288] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865), 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207)
I20260810 22:58:58.866705 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.863937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:48071 (local address 127.6.47.130:35865)
0810 22:58:58.863986 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.863988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.864102 (+ 114us) server_negotiation.cc:407] Connection header received
0810 22:58:58.864168 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.864169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.864182 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.864250 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:58.864458 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.865031 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866002 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.866347 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866428 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.866438 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.866584 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.866624 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 22:58:58.866734 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.863910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:45273 (local address 127.6.47.129:45207)
0810 22:58:58.863957 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.863959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.863971 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:58:58.864015 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.864016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.864028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.864086 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:58:58.864297 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.865009 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866005 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.866357 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866440 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.866449 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.866633 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.866672 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 22:58:58.866727 21117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.863838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:35865 (local address 127.6.47.131:48071)
0810 22:58:58.864048 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.864056 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.864117 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.864292 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.864293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.864294 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:58.864371 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.864373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.865143 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.865144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.865909 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.865911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.866429 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.866513 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.866516 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.866558 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.866604 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":41,"thread_start_us":98,"threads_started":1}
I20260810 22:58:58.866737 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.863839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45207 (local address 127.6.47.131:45273)
0810 22:58:58.863883 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:58.863896 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:58.863944 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:58.864114 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:58.864115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:58.864118 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:58:58.864208 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.864210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.865088 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.865089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.865904 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:58.865907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.866458 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.866459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:58.866533 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.866535 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 22:58:58.866608 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 22:58:58.866642 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":22}
I20260810 22:58:58.867201 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99ba3abbf4417db82f9c4024992c21" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf" is_pre_election: true
I20260810 22:58:58.867353 20737 raft_consensus.cc:2454] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:58.867311 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99ba3abbf4417db82f9c4024992c21" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df" is_pre_election: true
I20260810 22:58:58.867458 20498 raft_consensus.cc:2454] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:58.867885 20863 leader_election.cc:302] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:58.868021 21112 raft_consensus.cc:2781] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:58.868085 21112 raft_consensus.cc:482] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:58.868103 21112 raft_consensus.cc:3037] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:58.868834 21112 raft_consensus.cc:504] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.868990 21112 leader_election.cc:288] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Requested vote from peers 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865), 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207)
I20260810 22:58:58.869169 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99ba3abbf4417db82f9c4024992c21" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df"
I20260810 22:58:58.869230 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab99ba3abbf4417db82f9c4024992c21" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf"
I20260810 22:58:58.869297 20498 raft_consensus.cc:3037] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:58.869354 20737 raft_consensus.cc:3037] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:58.870019 20498 raft_consensus.cc:2454] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:58.870066 20737 raft_consensus.cc:2454] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:58.870239 20858 leader_election.cc:302] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0c3468e0cb42cbb8216476bbb080df, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:58.870342 21112 raft_consensus.cc:2781] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:58.870476 21112 raft_consensus.cc:686] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Becoming Leader. State: Replica: f69d60b471fd4aae939a09780c588832, State: Running, Role: LEADER
I20260810 22:58:58.870581 21112 consensus_queue.cc:237] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } }
I20260810 22:58:58.871451 20228 catalog_manager.cc:5495] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f69d60b471fd4aae939a09780c588832 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } health_report { overall_health: UNKNOWN } } }
I20260810 22:58:58.892705 21092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:58.890299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.6.47.131:37403)
0810 22:58:58.890343 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:58.890345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:58.890369 (+ 24us) server_negotiation.cc:407] Connection header received
0810 22:58:58.890403 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:58.890404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:58.890422 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:58.890483 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:58.890673 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.891258 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.892028 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:58.892203 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:58.892255 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:58.892404 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:58.892499 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:58.892591 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:58.892626 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 22:58:58.892643 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:58:58.894498 20498 raft_consensus.cc:1261] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:58.894634 20737 raft_consensus.cc:1261] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:58.894866 21112 consensus_queue.cc:1035] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:58.895043 21119 consensus_queue.cc:1035] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:58.897503 21124 mvcc.cc:205] Tried to move back new op lower bound from 7317105618509021184 to 7317105618414600192. Current Snapshot: MvccSnapshot[applied={T|T < 7317105618509021184}]
I20260810 22:58:58.899893 20228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:58.900007 20228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:58.900848 20228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Sending DeleteTablet for 3 replicas of tablet ab99ba3abbf4417db82f9c4024992c21
I20260810 22:58:58.901194 20946 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99ba3abbf4417db82f9c4024992c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:58 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260810 22:58:58.901211 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99ba3abbf4417db82f9c4024992c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:58 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260810 22:58:58.901261 20468 tablet_service.cc:1514] Processing DeleteTablet for tablet ab99ba3abbf4417db82f9c4024992c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:58 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:53062
I20260810 22:58:58.901418 21144 tablet_replica.cc:330] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: stopping tablet replica
I20260810 22:58:58.901510 21144 raft_consensus.cc:2229] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:58.901613 21144 raft_consensus.cc:2258] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:58.902050 21143 tablet_replica.cc:330] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: stopping tablet replica
I20260810 22:58:58.901983 20228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:58.902143 20228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:58.902156 21143 raft_consensus.cc:2229] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:58.902138 21144 ts_tablet_manager.cc:1905] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:58.902254 21143 raft_consensus.cc:2258] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:58.903458 20946 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 (DEFAULT_TABLE table=test-table [id=115518b0977b41b2843beacf60c98c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.903587 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd138e7584c44e2b2b00b1344afa404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.903609 21143 ts_tablet_manager.cc:1905] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:58.904763 21144 ts_tablet_manager.cc:1918] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:58.904851 21144 log.cc:1196] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/ab99ba3abbf4417db82f9c4024992c21
I20260810 22:58:58.905117 21144 ts_tablet_manager.cc:1939] T ab99ba3abbf4417db82f9c4024992c21 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting consensus metadata
I20260810 22:58:58.905114 21102 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Bootstrap starting.
I20260810 22:58:58.905489 20711 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 (DEFAULT_TABLE table=test-table [id=115518b0977b41b2843beacf60c98c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.905586 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd138e7584c44e2b2b00b1344afa404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.905613 21102 tablet_bootstrap.cc:654] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.906373 21102 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: No bootstrap required, opened a new log
I20260810 22:58:58.906426 21102 ts_tablet_manager.cc:1397] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260810 22:58:58.906585 21102 raft_consensus.cc:348] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.906646 21102 raft_consensus.cc:374] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.906665 21102 raft_consensus.cc:729] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69d60b471fd4aae939a09780c588832, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.906710 21102 consensus_queue.cc:260] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.906800 21102 ts_tablet_manager.cc:1428] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.906940 21108 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Bootstrap starting.
I20260810 22:58:58.907048 20200 catalog_manager.cc:4841] TS 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207): tablet ab99ba3abbf4417db82f9c4024992c21 (table test-table [id=37ae73ca04fa493ca260ddff4883c495]) successfully deleted
I20260810 22:58:58.907361 21108 tablet_bootstrap.cc:654] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.907358 20468 tablet_service.cc:1467] Processing CreateTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 (DEFAULT_TABLE table=test-table [id=115518b0977b41b2843beacf60c98c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:58.907487 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fd138e7584c44e2b2b00b1344afa404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:58.908020 21108 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: No bootstrap required, opened a new log
I20260810 22:58:58.908074 21108 ts_tablet_manager.cc:1397] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:58.908221 21108 raft_consensus.cc:348] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.908301 21108 raft_consensus.cc:374] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.908326 21108 raft_consensus.cc:729] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e38be1700c454d86402e456446a0cf, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.908376 21108 consensus_queue.cc:260] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.908468 21108 ts_tablet_manager.cc:1428] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:58.908742 21100 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Bootstrap starting.
I20260810 22:58:58.909144 21100 tablet_bootstrap.cc:654] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:58.909763 21143 ts_tablet_manager.cc:1918] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:58.909824 21143 log.cc:1196] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/ab99ba3abbf4417db82f9c4024992c21
I20260810 22:58:58.910004 21143 ts_tablet_manager.cc:1939] T ab99ba3abbf4417db82f9c4024992c21 P 21e38be1700c454d86402e456446a0cf: Deleting consensus metadata
I20260810 22:58:58.910158 21141 tablet_replica.cc:330] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: stopping tablet replica
I20260810 22:58:58.910240 21141 raft_consensus.cc:2229] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:58.910315 21141 raft_consensus.cc:2258] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:58.910732 21100 tablet_bootstrap.cc:492] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: No bootstrap required, opened a new log
I20260810 22:58:58.910790 21100 ts_tablet_manager.cc:1397] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:58:58.910808 20203 catalog_manager.cc:4841] TS 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865): tablet ab99ba3abbf4417db82f9c4024992c21 (table test-table [id=37ae73ca04fa493ca260ddff4883c495]) successfully deleted
I20260810 22:58:58.910934 21141 ts_tablet_manager.cc:1905] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:58.910953 21100 raft_consensus.cc:348] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.911022 21100 raft_consensus.cc:374] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:58.911044 21100 raft_consensus.cc:729] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0c3468e0cb42cbb8216476bbb080df, State: Initialized, Role: FOLLOWER
I20260810 22:58:58.911111 21100 consensus_queue.cc:260] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:58.911223 21100 ts_tablet_manager.cc:1428] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:58.912686 21141 ts_tablet_manager.cc:1918] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:58.912765 21141 log.cc:1196] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/ab99ba3abbf4417db82f9c4024992c21
I20260810 22:58:58.913021 21141 ts_tablet_manager.cc:1939] T ab99ba3abbf4417db82f9c4024992c21 P f69d60b471fd4aae939a09780c588832: Deleting consensus metadata
I20260810 22:58:58.913416 20198 catalog_manager.cc:4841] TS f69d60b471fd4aae939a09780c588832 (127.6.47.131:37403): tablet ab99ba3abbf4417db82f9c4024992c21 (table test-table [id=37ae73ca04fa493ca260ddff4883c495]) successfully deleted
I20260810 22:58:59.140729 21112 raft_consensus.cc:482] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:59.140839 21112 raft_consensus.cc:504] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.141021 21112 leader_election.cc:288] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.141288 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd138e7584c44e2b2b00b1344afa404" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf" is_pre_election: true
I20260810 22:58:59.141384 20737 raft_consensus.cc:2454] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.141307 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd138e7584c44e2b2b00b1344afa404" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df" is_pre_election: true
I20260810 22:58:59.141458 20498 raft_consensus.cc:2454] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.141676 20863 leader_election.cc:302] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.141825 21112 raft_consensus.cc:2781] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:59.141885 21112 raft_consensus.cc:482] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:59.141927 21112 raft_consensus.cc:3037] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.142617 21112 raft_consensus.cc:504] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.142768 21112 leader_election.cc:288] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Requested vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.142925 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd138e7584c44e2b2b00b1344afa404" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf"
I20260810 22:58:59.142927 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fd138e7584c44e2b2b00b1344afa404" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df"
I20260810 22:58:59.142997 20498 raft_consensus.cc:3037] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.143013 20737 raft_consensus.cc:3037] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.143612 20737 raft_consensus.cc:2454] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.143821 20863 leader_election.cc:302] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.143975 21112 raft_consensus.cc:2781] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:59.144052 21112 raft_consensus.cc:686] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Becoming Leader. State: Replica: f69d60b471fd4aae939a09780c588832, State: Running, Role: LEADER
I20260810 22:58:59.144177 20498 raft_consensus.cc:2454] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.144131 21112 consensus_queue.cc:237] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.144810 20226 catalog_manager.cc:5495] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f69d60b471fd4aae939a09780c588832 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } health_report { overall_health: HEALTHY } } }
Aug 10 22:58:59 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402738, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.190@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:59.216158 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.209384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.6.47.190:45251)
0810 22:58:59.209542 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.209544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.209578 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:58:59.209659 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.209660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.209670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.209737 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:59.210237 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.210732 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.211442 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.211645 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.211702 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.213851 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:59.213852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:59.213853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:59.213861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:59.215153 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:59.215513 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:59.215514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:59.215514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:59.215568 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:59.215751 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:59.215751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:59.215752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:59.215850 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:59.215899 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:59.216019 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 22:58:59.216088 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260810 22:58:59.216262 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.209301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:45251 (local address 127.0.0.1:38500)
0810 22:58:59.209519 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:59.209529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:59.209595 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:59.209759 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:59.209760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:59.210072 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:58:59.210159 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.210162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.210802 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.210803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.211365 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.211366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.211715 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.211716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.211755 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.212335 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:58:59.212340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:58:59.213754 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:58:59.215241 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:59.215249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:59.215251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:59.215448 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:59.215629 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:58:59.215630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:58:59.215630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:58:59.215688 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:58:59.215904 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:58:59.215910 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:58:59.215952 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 22:58:59.216003 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 22:58:59.216169 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":75,"thread_start_us":98,"threads_started":1}
I20260810 22:58:59.218348 20226 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:38500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 22:58:59.218603 20226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:59.220382 20711 tablet_service.cc:1467] Processing CreateTablet for tablet 098ed4793dff426faaaa1642f59b1f65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2551efa95f5e4bd188541aa6d1e3e3d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:59.220403 20468 tablet_service.cc:1467] Processing CreateTablet for tablet 098ed4793dff426faaaa1642f59b1f65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2551efa95f5e4bd188541aa6d1e3e3d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:59.220571 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ed4793dff426faaaa1642f59b1f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.220696 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ed4793dff426faaaa1642f59b1f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.221930 21108 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: Bootstrap starting.
I20260810 22:58:59.222185 21100 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: Bootstrap starting.
I20260810 22:58:59.222291 20946 tablet_service.cc:1467] Processing CreateTablet for tablet 098ed4793dff426faaaa1642f59b1f65 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2551efa95f5e4bd188541aa6d1e3e3d0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 22:58:59.222450 21108 tablet_bootstrap.cc:654] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.222508 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098ed4793dff426faaaa1642f59b1f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.222676 21100 tablet_bootstrap.cc:654] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.223189 21108 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: No bootstrap required, opened a new log
I20260810 22:58:59.223258 21108 ts_tablet_manager.cc:1397] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:59.223354 21100 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: No bootstrap required, opened a new log
I20260810 22:58:59.223408 21100 ts_tablet_manager.cc:1397] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.223444 21108 raft_consensus.cc:348] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.223552 21108 raft_consensus.cc:374] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.223582 21108 raft_consensus.cc:729] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e38be1700c454d86402e456446a0cf, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.223603 21100 raft_consensus.cc:348] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.223680 21100 raft_consensus.cc:374] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.223654 21108 consensus_queue.cc:260] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.223708 21100 raft_consensus.cc:729] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0c3468e0cb42cbb8216476bbb080df, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.223758 21108 ts_tablet_manager.cc:1428] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 22:58:59.223755 21100 consensus_queue.cc:260] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.223855 21100 ts_tablet_manager.cc:1428] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:59.223904 21115 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.224002 21115 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.224298 21102 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: Bootstrap starting.
I20260810 22:58:59.224399 21111 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.224565 21111 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.224746 21102 tablet_bootstrap.cc:654] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.225431 21102 tablet_bootstrap.cc:492] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: No bootstrap required, opened a new log
I20260810 22:58:59.225498 21102 ts_tablet_manager.cc:1397] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.225626 21102 raft_consensus.cc:348] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.225698 21102 raft_consensus.cc:374] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.225721 21102 raft_consensus.cc:729] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69d60b471fd4aae939a09780c588832, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.225769 21102 consensus_queue.cc:260] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.225859 21102 ts_tablet_manager.cc:1428] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:59.225883 21112 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.226022 21112 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.370520 21112 raft_consensus.cc:482] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:59.370620 21112 raft_consensus.cc:504] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.370819 21112 leader_election.cc:288] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.371039 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ed4793dff426faaaa1642f59b1f65" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf" is_pre_election: true
I20260810 22:58:59.371039 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ed4793dff426faaaa1642f59b1f65" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df" is_pre_election: true
I20260810 22:58:59.371145 20737 raft_consensus.cc:2454] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.371146 20498 raft_consensus.cc:2454] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.371315 20863 leader_election.cc:302] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.371443 21112 raft_consensus.cc:2781] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:59.371496 21112 raft_consensus.cc:482] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:59.371527 21112 raft_consensus.cc:3037] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.372236 21112 raft_consensus.cc:504] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.372419 21112 leader_election.cc:288] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Requested vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.372589 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ed4793dff426faaaa1642f59b1f65" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf"
I20260810 22:58:59.372609 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098ed4793dff426faaaa1642f59b1f65" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df"
I20260810 22:58:59.372668 20498 raft_consensus.cc:3037] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.372694 20737 raft_consensus.cc:3037] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.373270 20737 raft_consensus.cc:2454] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.373302 20498 raft_consensus.cc:2454] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.373512 20858 leader_election.cc:302] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, 4a0c3468e0cb42cbb8216476bbb080df, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.373632 21112 raft_consensus.cc:2781] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:59.373703 21112 raft_consensus.cc:686] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Becoming Leader. State: Replica: f69d60b471fd4aae939a09780c588832, State: Running, Role: LEADER
I20260810 22:58:59.373761 21112 consensus_queue.cc:237] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.374032 21119 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: TxnStatusTablet state changed. Reason: New leader f69d60b471fd4aae939a09780c588832. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.374150 21119 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:59.374418 21174 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 22:58:59.374457 20228 catalog_manager.cc:5495] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f69d60b471fd4aae939a09780c588832 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } health_report { overall_health: HEALTHY } } }
I20260810 22:58:59.374564 21174 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 22:58:59.407176 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.404667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37403 (local address 127.0.0.1:43686)
0810 22:58:59.404749 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:59.404759 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:59.404816 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:59.404995 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:59.404996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:59.404997 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:59.405048 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.405052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.405738 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.405740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.406364 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.406367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.406733 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.406735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.406795 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.406811 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:59.407008 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:59.407018 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 22:58:59.407060 (+ 42us) client_negotiation.cc:240] Negotiation successful
0810 22:58:59.407111 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 22:58:59.407174 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.404727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.6.47.131:37403)
0810 22:58:59.404844 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.404845 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.404854 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:58:59.404883 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.404884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.404895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.404984 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:59.405139 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.405643 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.406441 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.406633 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.406704 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.406894 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:59.406937 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:59.406993 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:59.407061 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 22:58:59.407082 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260810 22:58:59.408299 20737 raft_consensus.cc:1261] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.408300 20498 raft_consensus.cc:1261] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.408550 21112 consensus_queue.cc:1035] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.408744 21112 consensus_queue.cc:1035] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.409107 21111 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: TxnStatusTablet state changed. Reason: New leader f69d60b471fd4aae939a09780c588832. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.409188 21111 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.409227 21115 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: TxnStatusTablet state changed. Reason: New leader f69d60b471fd4aae939a09780c588832. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.409346 21115 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.409813 21119 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.409893 21119 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:59.410043 21111 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.410133 21111 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 4a0c3468e0cb42cbb8216476bbb080df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.410307 21184 mvcc.cc:205] Tried to move back new op lower bound from 7317105620614217728 to 7317105620475260928. Current Snapshot: MvccSnapshot[applied={T|T < 7317105620614217728}]
I20260810 22:58:59.411288 21112 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.411365 21112 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P f69d60b471fd4aae939a09780c588832: This TxnStatusTablet replica's current role is: LEADER
I20260810 22:58:59.411643 21186 mvcc.cc:205] Tried to move back new op lower bound from 7317105620614217728 to 7317105620475260928. Current Snapshot: MvccSnapshot[applied={T|T < 7317105620614217728}]
I20260810 22:58:59.411646 21115 tablet_replica.cc:439] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } } }
I20260810 22:58:59.411798 21115 tablet_replica.cc:442] T 098ed4793dff426faaaa1642f59b1f65 P 21e38be1700c454d86402e456446a0cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 22:58:59.416630 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.414362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:35917 (local address 127.6.47.131:37403)
0810 22:58:59.414465 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.414467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.414521 (+ 54us) server_negotiation.cc:407] Connection header received
0810 22:58:59.414592 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.414592 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.414601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.414653 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:59.414776 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.415207 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.415946 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.416146 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.416217 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.416389 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:59.416418 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:59.416463 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:59.416531 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 22:58:59.416552 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":87}
I20260810 22:58:59.416651 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.414329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:37403 (local address 127.6.47.131:35917)
0810 22:58:59.414471 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:59.414482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:58:59.414537 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:58:59.414654 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:58:59.414655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:58:59.414656 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 22:58:59.414714 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.414715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.415287 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.415288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.415838 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:58:59.415839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.416214 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.416215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:58:59.416273 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.416288 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 22:58:59.416463 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 22:58:59.416464 (+ 1us) client_negotiation.cc:769] Sending connection context
0810 22:58:59.416531 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:58:59.416569 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260810 22:58:59.418897 20498 raft_consensus.cc:1261] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.418903 20737 raft_consensus.cc:1261] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.419101 21112 consensus_queue.cc:1035] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.419214 21119 consensus_queue.cc:1035] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.431948 20228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.432037 20228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.432717 20228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Sending DeleteTablet for 3 replicas of tablet 9fd138e7584c44e2b2b00b1344afa404
I20260810 22:58:59.432972 20946 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260810 22:58:59.433012 20468 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:53062
I20260810 22:58:59.433033 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fd138e7584c44e2b2b00b1344afa404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260810 22:58:59.433228 21206 tablet_replica.cc:330] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: stopping tablet replica
I20260810 22:58:59.433358 21205 tablet_replica.cc:330] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: stopping tablet replica
I20260810 22:58:59.433436 21205 raft_consensus.cc:2229] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.433511 21205 raft_consensus.cc:2258] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.433599 21206 raft_consensus.cc:2229] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:59.433699 21206 raft_consensus.cc:2258] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.433930 21205 ts_tablet_manager.cc:1905] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.434022 21206 ts_tablet_manager.cc:1905] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.435746 21205 ts_tablet_manager.cc:1918] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:59.435866 21205 log.cc:1196] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/9fd138e7584c44e2b2b00b1344afa404
I20260810 22:58:59.435922 21206 ts_tablet_manager.cc:1918] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:59.435992 21206 log.cc:1196] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/9fd138e7584c44e2b2b00b1344afa404
I20260810 22:58:59.436269 21206 ts_tablet_manager.cc:1939] T 9fd138e7584c44e2b2b00b1344afa404 P f69d60b471fd4aae939a09780c588832: Deleting consensus metadata
I20260810 22:58:59.436271 21205 ts_tablet_manager.cc:1939] T 9fd138e7584c44e2b2b00b1344afa404 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting consensus metadata
I20260810 22:58:59.436620 20198 catalog_manager.cc:4841] TS f69d60b471fd4aae939a09780c588832 (127.6.47.131:37403): tablet 9fd138e7584c44e2b2b00b1344afa404 (table test-table [id=115518b0977b41b2843beacf60c98c67]) successfully deleted
I20260810 22:58:59.436669 20200 catalog_manager.cc:4841] TS 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207): tablet 9fd138e7584c44e2b2b00b1344afa404 (table test-table [id=115518b0977b41b2843beacf60c98c67]) successfully deleted
I20260810 22:58:59.437731 21208 tablet_replica.cc:330] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: stopping tablet replica
I20260810 22:58:59.437783 21208 raft_consensus.cc:2229] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.437875 21208 raft_consensus.cc:2258] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.438081 21208 ts_tablet_manager.cc:1905] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.439296 21208 ts_tablet_manager.cc:1918] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:59.439342 21208 log.cc:1196] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/9fd138e7584c44e2b2b00b1344afa404
I20260810 22:58:59.439548 21208 ts_tablet_manager.cc:1939] T 9fd138e7584c44e2b2b00b1344afa404 P 21e38be1700c454d86402e456446a0cf: Deleting consensus metadata
I20260810 22:58:59.439875 20203 catalog_manager.cc:4841] TS 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865): tablet 9fd138e7584c44e2b2b00b1344afa404 (table test-table [id=115518b0977b41b2843beacf60c98c67]) successfully deleted
Aug 10 22:58:59 dist-test-slave-rwrg krb5kdc[20155](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 22:58:59 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:58:59 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:58:59.462688 21165 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:58:59.456604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.6.47.190:45251)
0810 22:58:59.456651 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.456653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.456664 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:58:59.456721 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.456722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.456734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.456807 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:59.457500 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.458269 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.459139 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.459443 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.459558 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.462603 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38504: BlockingRecv error: recv got EOF from 127.0.0.1:38504 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 10 22:58:59 dist-test-slave-rwrg krb5kdc[20155](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM
I20260810 22:58:59.470794 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.463558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38516 (local address 127.6.47.190:45251)
0810 22:58:59.463614 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.463616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.463719 (+ 103us) server_negotiation.cc:407] Connection header received
0810 22:58:59.463811 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.463812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.463822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.463892 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:58:59.464403 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.464955 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.465623 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.465825 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.465909 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.468004 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:58:59.468006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:58:59.468006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:58:59.468013 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:58:59.469411 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:59.469786 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:59.469787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:59.469787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:59.469848 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:58:59.470059 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:58:59.470060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:58:59.470060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:58:59.470228 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:58:59.470329 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:59.470538 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 22:58:59.470642 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:58:59.472694 20228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:59.472882 20228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:59.474287 20468 tablet_service.cc:1467] Processing CreateTablet for tablet c560340064d64141a26618b08cfe0b46 (DEFAULT_TABLE table=test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.474324 20946 tablet_service.cc:1467] Processing CreateTablet for tablet c560340064d64141a26618b08cfe0b46 (DEFAULT_TABLE table=test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.474445 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c560340064d64141a26618b08cfe0b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.474445 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c560340064d64141a26618b08cfe0b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.475775 21100 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Bootstrap starting.
I20260810 22:58:59.475811 20711 tablet_service.cc:1467] Processing CreateTablet for tablet c560340064d64141a26618b08cfe0b46 (DEFAULT_TABLE table=test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.475943 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c560340064d64141a26618b08cfe0b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.476210 21100 tablet_bootstrap.cc:654] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.476964 21100 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: No bootstrap required, opened a new log
I20260810 22:58:59.477020 21100 ts_tablet_manager.cc:1397] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.477173 21100 raft_consensus.cc:348] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.477248 21100 raft_consensus.cc:374] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.477268 21100 raft_consensus.cc:729] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0c3468e0cb42cbb8216476bbb080df, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.477296 21100 consensus_queue.cc:260] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.477350 21102 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Bootstrap starting.
I20260810 22:58:59.477381 21100 ts_tablet_manager.cc:1428] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:59.477473 21108 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Bootstrap starting.
I20260810 22:58:59.477669 21102 tablet_bootstrap.cc:654] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.477869 21108 tablet_bootstrap.cc:654] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.478322 21102 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: No bootstrap required, opened a new log
I20260810 22:58:59.478374 21102 ts_tablet_manager.cc:1397] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.478535 21102 raft_consensus.cc:348] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.478598 21102 raft_consensus.cc:374] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.478639 21102 raft_consensus.cc:729] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69d60b471fd4aae939a09780c588832, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.478688 21108 tablet_bootstrap.cc:492] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: No bootstrap required, opened a new log
I20260810 22:58:59.478683 21102 consensus_queue.cc:260] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.478730 21108 ts_tablet_manager.cc:1397] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 22:58:59.478786 21102 ts_tablet_manager.cc:1428] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:59.478871 21108 raft_consensus.cc:348] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.478940 21108 raft_consensus.cc:374] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.478969 21108 raft_consensus.cc:729] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e38be1700c454d86402e456446a0cf, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.479008 21108 consensus_queue.cc:260] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.479091 21108 ts_tablet_manager.cc:1428] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:59.702911 21193 raft_consensus.cc:482] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:59.703038 21193 raft_consensus.cc:504] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.703261 21193 leader_election.cc:288] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.703507 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c560340064d64141a26618b08cfe0b46" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df" is_pre_election: true
I20260810 22:58:59.703507 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c560340064d64141a26618b08cfe0b46" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf" is_pre_election: true
I20260810 22:58:59.703624 20737 raft_consensus.cc:2454] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.703624 20498 raft_consensus.cc:2454] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.703799 20863 leader_election.cc:302] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.703908 21193 raft_consensus.cc:2781] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:59.703951 21193 raft_consensus.cc:482] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:59.703970 21193 raft_consensus.cc:3037] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.704768 21193 raft_consensus.cc:504] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.704929 21193 leader_election.cc:288] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Requested vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.705124 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c560340064d64141a26618b08cfe0b46" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df"
I20260810 22:58:59.705200 20498 raft_consensus.cc:3037] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.706090 20498 raft_consensus.cc:2454] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.706261 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c560340064d64141a26618b08cfe0b46" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf"
I20260810 22:58:59.706351 20737 raft_consensus.cc:3037] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.706338 20858 leader_election.cc:302] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0c3468e0cb42cbb8216476bbb080df, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.706439 21193 raft_consensus.cc:2781] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:59.706528 21193 raft_consensus.cc:686] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Becoming Leader. State: Replica: f69d60b471fd4aae939a09780c588832, State: Running, Role: LEADER
I20260810 22:58:59.706633 21193 consensus_queue.cc:237] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.707204 20737 raft_consensus.cc:2454] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.707804 20226 catalog_manager.cc:5495] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f69d60b471fd4aae939a09780c588832 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } health_report { overall_health: HEALTHY } } }
I20260810 22:58:59.762115 21180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.760031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.6.47.131:37403)
0810 22:58:59.760092 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 22:58:59.760094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:58:59.760101 (+ 7us) server_negotiation.cc:407] Connection header received
0810 22:58:59.760131 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:58:59.760132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:58:59.760145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:58:59.760196 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:58:59.760337 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.760825 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.761503 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:58:59.761681 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:58:59.761733 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:58:59.761857 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:58:59.761883 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:58:59.761932 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 22:58:59.762019 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 22:58:59.762036 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 22:58:59.763029 20737 raft_consensus.cc:1261] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.763079 20498 raft_consensus.cc:1261] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.763337 21193 consensus_queue.cc:1035] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.763450 21112 consensus_queue.cc:1035] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.766191 20226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.766316 20226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.766886 20226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Sending DeleteTablet for 3 replicas of tablet c560340064d64141a26618b08cfe0b46
I20260810 22:58:59.767225 20946 tablet_service.cc:1514] Processing DeleteTablet for tablet c560340064d64141a26618b08cfe0b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260810 22:58:59.767225 20468 tablet_service.cc:1514] Processing DeleteTablet for tablet c560340064d64141a26618b08cfe0b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:53062
I20260810 22:58:59.767326 21206 tablet_replica.cc:330] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: stopping tablet replica
I20260810 22:58:59.767331 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet c560340064d64141a26618b08cfe0b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260810 22:58:59.767342 21205 tablet_replica.cc:330] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: stopping tablet replica
I20260810 22:58:59.767390 21206 raft_consensus.cc:2229] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:59.767405 21208 tablet_replica.cc:330] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: stopping tablet replica
I20260810 22:58:59.767421 21205 raft_consensus.cc:2229] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.767447 21208 raft_consensus.cc:2229] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.767453 21206 raft_consensus.cc:2258] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.767475 21205 raft_consensus.cc:2258] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.767508 21208 raft_consensus.cc:2258] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.767647 20226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:58:59.767798 20226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:58:59.767809 21206 ts_tablet_manager.cc:1905] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.767867 21205 ts_tablet_manager.cc:1905] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.768812 21208 ts_tablet_manager.cc:1905] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.769487 21206 ts_tablet_manager.cc:1918] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:59.769487 21205 ts_tablet_manager.cc:1918] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:59.769543 21206 log.cc:1196] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/c560340064d64141a26618b08cfe0b46
I20260810 22:58:59.769543 21205 log.cc:1196] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/c560340064d64141a26618b08cfe0b46
I20260810 22:58:59.769774 21206 ts_tablet_manager.cc:1939] T c560340064d64141a26618b08cfe0b46 P f69d60b471fd4aae939a09780c588832: Deleting consensus metadata
I20260810 22:58:59.769795 21205 ts_tablet_manager.cc:1939] T c560340064d64141a26618b08cfe0b46 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting consensus metadata
I20260810 22:58:59.770104 21208 ts_tablet_manager.cc:1918] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:58:59.770156 21208 log.cc:1196] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/c560340064d64141a26618b08cfe0b46
I20260810 22:58:59.770123 20198 catalog_manager.cc:4841] TS f69d60b471fd4aae939a09780c588832 (127.6.47.131:37403): tablet c560340064d64141a26618b08cfe0b46 (table test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]) successfully deleted
I20260810 22:58:59.770134 20200 catalog_manager.cc:4841] TS 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207): tablet c560340064d64141a26618b08cfe0b46 (table test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]) successfully deleted
I20260810 22:58:59.770393 21208 ts_tablet_manager.cc:1939] T c560340064d64141a26618b08cfe0b46 P 21e38be1700c454d86402e456446a0cf: Deleting consensus metadata
I20260810 22:58:59.770421 20468 tablet_service.cc:1467] Processing CreateTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 (DEFAULT_TABLE table=test-table [id=0c543e81c23a4fe7a723c77b7767df6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.770443 20711 tablet_service.cc:1467] Processing CreateTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 (DEFAULT_TABLE table=test-table [id=0c543e81c23a4fe7a723c77b7767df6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.770443 20946 tablet_service.cc:1467] Processing CreateTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 (DEFAULT_TABLE table=test-table [id=0c543e81c23a4fe7a723c77b7767df6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:58:59.770517 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cb76bf562745678a29d18ca12b1dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.770547 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cb76bf562745678a29d18ca12b1dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.770547 20946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37cb76bf562745678a29d18ca12b1dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:58:59.771401 21100 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: Bootstrap starting.
I20260810 22:58:59.771714 21108 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Bootstrap starting.
I20260810 22:58:59.771728 21102 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Bootstrap starting.
I20260810 22:58:59.772061 21102 tablet_bootstrap.cc:654] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.772117 21100 tablet_bootstrap.cc:654] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.772404 21108 tablet_bootstrap.cc:654] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Neither blocks nor log segments found. Creating new log.
I20260810 22:58:59.772743 21102 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: No bootstrap required, opened a new log
I20260810 22:58:59.772789 21100 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: No bootstrap required, opened a new log
I20260810 22:58:59.772795 21102 ts_tablet_manager.cc:1397] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.772840 21100 ts_tablet_manager.cc:1397] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260810 22:58:59.772946 21102 raft_consensus.cc:348] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773026 21102 raft_consensus.cc:374] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.773049 21102 raft_consensus.cc:729] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69d60b471fd4aae939a09780c588832, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.773064 20203 catalog_manager.cc:4841] TS 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865): tablet c560340064d64141a26618b08cfe0b46 (table test-table [id=bcf663d4f15b40089b6b1e6d837e4cc5]) successfully deleted
I20260810 22:58:59.772996 21100 raft_consensus.cc:348] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773090 21102 consensus_queue.cc:260] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773116 21100 raft_consensus.cc:374] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.773144 21100 raft_consensus.cc:729] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0c3468e0cb42cbb8216476bbb080df, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.773192 21102 ts_tablet_manager.cc:1428] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:59.773192 21100 consensus_queue.cc:260] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773278 21108 tablet_bootstrap.cc:492] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: No bootstrap required, opened a new log
I20260810 22:58:59.773286 21100 ts_tablet_manager.cc:1428] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 22:58:59.773324 21108 ts_tablet_manager.cc:1397] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 22:58:59.773468 21108 raft_consensus.cc:348] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773541 21108 raft_consensus.cc:374] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:58:59.773569 21108 raft_consensus.cc:729] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21e38be1700c454d86402e456446a0cf, State: Initialized, Role: FOLLOWER
I20260810 22:58:59.773614 21108 consensus_queue.cc:260] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.773716 21108 ts_tablet_manager.cc:1428] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260810 22:58:59.859733 21119 raft_consensus.cc:482] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:58:59.859856 21119 raft_consensus.cc:504] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.860093 21119 leader_election.cc:288] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.860368 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cb76bf562745678a29d18ca12b1dc5" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df" is_pre_election: true
I20260810 22:58:59.860381 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cb76bf562745678a29d18ca12b1dc5" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf" is_pre_election: true
I20260810 22:58:59.860476 20737 raft_consensus.cc:2454] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.860486 20498 raft_consensus.cc:2454] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 0.
I20260810 22:58:59.860695 20863 leader_election.cc:302] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.860807 21119 raft_consensus.cc:2781] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:58:59.860850 21119 raft_consensus.cc:482] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:58:59.860881 21119 raft_consensus.cc:3037] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.861620 21119 raft_consensus.cc:504] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.861773 21119 leader_election.cc:288] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Requested vote from peers 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207), 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865)
I20260810 22:58:59.861923 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cb76bf562745678a29d18ca12b1dc5" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21e38be1700c454d86402e456446a0cf"
I20260810 22:58:59.861999 20737 raft_consensus.cc:3037] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.861977 20498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37cb76bf562745678a29d18ca12b1dc5" candidate_uuid: "f69d60b471fd4aae939a09780c588832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0c3468e0cb42cbb8216476bbb080df"
I20260810 22:58:59.862051 20498 raft_consensus.cc:3037] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:58:59.862708 20737 raft_consensus.cc:2454] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.862927 20498 raft_consensus.cc:2454] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f69d60b471fd4aae939a09780c588832 in term 1.
I20260810 22:58:59.862957 20863 leader_election.cc:302] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21e38be1700c454d86402e456446a0cf, f69d60b471fd4aae939a09780c588832; no voters:
I20260810 22:58:59.863049 21119 raft_consensus.cc:2781] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:58:59.863144 21119 raft_consensus.cc:686] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Becoming Leader. State: Replica: f69d60b471fd4aae939a09780c588832, State: Running, Role: LEADER
I20260810 22:58:59.863214 21119 consensus_queue.cc:237] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } }
I20260810 22:58:59.863783 20228 catalog_manager.cc:5495] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 reported cstate change: term changed from 0 to 1, leader changed from <none> to f69d60b471fd4aae939a09780c588832 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "f69d60b471fd4aae939a09780c588832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f69d60b471fd4aae939a09780c588832" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 37403 } health_report { overall_health: HEALTHY } } }
I20260810 22:58:59.892867 20737 raft_consensus.cc:1261] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.892910 20498 raft_consensus.cc:1261] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Refusing update from remote peer f69d60b471fd4aae939a09780c588832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:58:59.893077 21119 consensus_queue.cc:1035] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21e38be1700c454d86402e456446a0cf" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.893178 21112 consensus_queue.cc:1035] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0c3468e0cb42cbb8216476bbb080df" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:58:59.902894 20228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.902997 20228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:58:59.903744 20228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4150231bf940ee9d2790e19c96def5: Sending DeleteTablet for 3 replicas of tablet 37cb76bf562745678a29d18ca12b1dc5
I20260810 22:58:59.904017 20468 tablet_service.cc:1514] Processing DeleteTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:53062
I20260810 22:58:59.904052 20946 tablet_service.cc:1514] Processing DeleteTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43682
I20260810 22:58:59.904124 21205 tablet_replica.cc:330] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: stopping tablet replica
I20260810 22:58:59.904177 21205 raft_consensus.cc:2229] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.904181 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet 37cb76bf562745678a29d18ca12b1dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:58:59 UTC) from {username='oryx', principal='oryx/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260810 22:58:59.904215 21205 raft_consensus.cc:2258] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.904282 21206 tablet_replica.cc:330] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: stopping tablet replica
I20260810 22:58:59.904289 21208 tablet_replica.cc:330] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: stopping tablet replica
I20260810 22:58:59.904341 21206 raft_consensus.cc:2229] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 1 LEADER]: Raft consensus shutting down.
I20260810 22:58:59.904347 21208 raft_consensus.cc:2229] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:58:59.904382 21208 raft_consensus.cc:2258] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.904402 21206 raft_consensus.cc:2258] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:58:59.904678 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 20348
I20260810 22:58:59.904784 21205 ts_tablet_manager.cc:1905] T 37cb76bf562745678a29d18ca12b1dc5 P 4a0c3468e0cb42cbb8216476bbb080df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.904831 21208 ts_tablet_manager.cc:1905] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.907862 21208 ts_tablet_manager.cc:1918] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:59.907931 21208 log.cc:1196] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/37cb76bf562745678a29d18ca12b1dc5
I20260810 22:58:59.908142 21208 ts_tablet_manager.cc:1939] T 37cb76bf562745678a29d18ca12b1dc5 P 21e38be1700c454d86402e456446a0cf: Deleting consensus metadata
I20260810 22:58:59.908469 20203 catalog_manager.cc:4841] TS 21e38be1700c454d86402e456446a0cf (127.6.47.130:35865): tablet 37cb76bf562745678a29d18ca12b1dc5 (table test-table [id=0c543e81c23a4fe7a723c77b7767df6e]) successfully deleted
I20260810 22:58:59.917662 21206 ts_tablet_manager.cc:1905] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:58:59.918987 21206 ts_tablet_manager.cc:1918] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 22:58:59.919046 21206 log.cc:1196] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/37cb76bf562745678a29d18ca12b1dc5
I20260810 22:58:59.919266 21206 ts_tablet_manager.cc:1939] T 37cb76bf562745678a29d18ca12b1dc5 P f69d60b471fd4aae939a09780c588832: Deleting consensus metadata
I20260810 22:58:59.919656 20198 catalog_manager.cc:4841] TS f69d60b471fd4aae939a09780c588832 (127.6.47.131:37403): tablet 37cb76bf562745678a29d18ca12b1dc5 (table test-table [id=0c543e81c23a4fe7a723c77b7767df6e]) successfully deleted
W20260810 22:58:59.922185 20200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:45207 (error 108)
I20260810 22:58:59.922562 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 20596
I20260810 22:58:59.922802 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:58:59.922585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45207 (local address 127.0.0.1:53066)
0810 22:58:59.922696 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 22:58:59.922754 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
W20260810 22:58:59.925345 20200 catalog_manager.cc:4584] TS 4a0c3468e0cb42cbb8216476bbb080df (127.6.47.129:45207): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37cb76bf562745678a29d18ca12b1dc5: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45207: connect: Connection refused (error 111)
I20260810 22:58:59.933549 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 20831
I20260810 22:58:59.944638 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 20174
2026-08-10T22:58:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3393 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:59:00 dist-test-slave-rwrg krb5kdc[21268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:59:00 dist-test-slave-rwrg krb5kdc[21268](info): set up 2 sockets
Aug 10 22:59:00 dist-test-slave-rwrg krb5kdc[21268](info): commencing operation
krb5kdc: starting...
W20260810 22:59:01.229934 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:59:01 dist-test-slave-rwrg krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:59:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:59:01.359558 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37959
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:37959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:01.442724 21292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:01.442970 21292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:01.443008 21292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:01.444132 21292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:01.444228 21292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:01.444252 21292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:01.444278 21292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:01.444306 21292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:01.445807 21292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:37959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:37959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:01.446082 21292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:01.446301 21292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:01.451104 21309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:01.451164 21306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:01.451105 21301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:01.451303 21292 server_base.cc:1034] running on GCE node
I20260810 22:59:01.451678 21292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:01.451964 21292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:01.453114 21292 hybrid_clock.cc:648] HybridClock initialized: now 1786402741453073 us; error 53 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T22:59:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1514 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:59:01 dist-test-slave-rwrg krb5kdc[21318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:59:01 dist-test-slave-rwrg krb5kdc[21318](info): set up 2 sockets
Aug 10 22:59:01 dist-test-slave-rwrg krb5kdc[21318](info): commencing operation
krb5kdc: starting...
W20260810 22:59:02.752882 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:59:02 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:59:02.871188 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41851
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:02.955554 21337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:02.955785 21337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:02.955822 21337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:02.956988 21337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:02.957049 21337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:02.957072 21337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:02.957125 21337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:02.957150 21337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:02.958616 21337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:02.958921 21337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:02.959153 21337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:02.964304 21348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:02.964298 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:02.964349 21337 server_base.cc:1034] running on GCE node
W20260810 22:59:02.964298 21350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:02.964861 21337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:02.965166 21337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:02.966333 21337 hybrid_clock.cc:648] HybridClock initialized: now 1786402742966326 us; error 37 us; skew 500 ppm
Aug 10 22:59:02 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402742, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:02.968870 21337 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:59:02.969374 21337 webserver.cc:460] Webserver started at http://127.6.47.190:35225/ using document root <none> and password file <none>
I20260810 22:59:02.969625 21337 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:02.969696 21337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:02.969844 21337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:02.970893 21337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "f50edd59ca6f4347811adc1f6ab918f6"
format_stamp: "Formatted at 2026-08-10 22:59:02 on dist-test-slave-rwrg"
server_key: "f8cc2649ced70f4f20e3210aa3d334ea"
server_key_iv: "827016379992f14a3a9ce52a55b67bce"
server_key_version: "encryptionkey@0"
I20260810 22:59:02.971259 21337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "f50edd59ca6f4347811adc1f6ab918f6"
format_stamp: "Formatted at 2026-08-10 22:59:02 on dist-test-slave-rwrg"
server_key: "f8cc2649ced70f4f20e3210aa3d334ea"
server_key_iv: "827016379992f14a3a9ce52a55b67bce"
server_key_version: "encryptionkey@0"
I20260810 22:59:02.972608 21337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 22:59:02.973369 21362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:02.973543 21337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:02.973649 21337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "f50edd59ca6f4347811adc1f6ab918f6"
format_stamp: "Formatted at 2026-08-10 22:59:02 on dist-test-slave-rwrg"
server_key: "f8cc2649ced70f4f20e3210aa3d334ea"
server_key_iv: "827016379992f14a3a9ce52a55b67bce"
server_key_version: "encryptionkey@0"
I20260810 22:59:02.973734 21337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.003684 21337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.006284 21337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.006464 21337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.010845 21337 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:41851
I20260810 22:59:03.010882 21446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:41851 every 8 connection(s)
I20260810 22:59:03.011272 21337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:59:03.012178 21447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:03.014561 21447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Bootstrap starting.
I20260810 22:59:03.015194 21447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:03.015511 21447 log.cc:824] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:03.016219 21447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: No bootstrap required, opened a new log
I20260810 22:59:03.016948 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 21337
I20260810 22:59:03.017062 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:59:03.017258 6334 external_mini_cluster.cc:1428] Setting key d2e60c63e4fd25650ac90b2089f91ec0
I20260810 22:59:03.018491 21447 raft_consensus.cc:348] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.018679 21447 raft_consensus.cc:374] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:03.018778 21447 raft_consensus.cc:729] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50edd59ca6f4347811adc1f6ab918f6, State: Initialized, Role: FOLLOWER
I20260810 22:59:03.018957 21447 consensus_queue.cc:260] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.019061 21447 raft_consensus.cc:388] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:59:03.019098 21447 raft_consensus.cc:482] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:59:03.019152 21447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:03.019794 21447 raft_consensus.cc:504] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.019886 21447 leader_election.cc:302] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f50edd59ca6f4347811adc1f6ab918f6; no voters:
I20260810 22:59:03.020100 21447 leader_election.cc:288] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:59:03.020169 21457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:59:03.020371 21447 sys_catalog.cc:564] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:59:03.020391 21457 raft_consensus.cc:686] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 LEADER]: Becoming Leader. State: Replica: f50edd59ca6f4347811adc1f6ab918f6, State: Running, Role: LEADER
I20260810 22:59:03.020632 21457 consensus_queue.cc:237] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.021003 21458 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f50edd59ca6f4347811adc1f6ab918f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } } }
I20260810 22:59:03.021068 21458 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:03.021250 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f50edd59ca6f4347811adc1f6ab918f6. Latest consensus state: current_term: 1 leader_uuid: "f50edd59ca6f4347811adc1f6ab918f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } } }
I20260810 22:59:03.021406 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:03.021430 21467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:59:03.021986 21467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:59:03.023659 21467 catalog_manager.cc:1347] Generated new cluster ID: 2a6e55ef802d425bac470c376dced5dc
I20260810 22:59:03.023721 21467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:03.029964 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.017812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.6.47.190:41851)
0810 22:59:03.017999 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.018007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.018033 (+ 26us) server_negotiation.cc:407] Connection header received
0810 22:59:03.018470 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.018477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.018523 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.018645 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.019444 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.021196 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.022403 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.022994 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.023110 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.026024 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:03.026028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:03.026029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:03.026041 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:03.028107 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.028539 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.028543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.028543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.028693 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.028909 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.028910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.028910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.029277 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:03.029498 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:03.029584 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 22:59:03.029753 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260810 22:59:03.044526 21467 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:59:03.045087 21467 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:59:03.056988 21467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Generated new TSK 0
I20260810 22:59:03.057201 21467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:59:03.091375 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:03.174015 21495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:03.174206 21495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:03.174228 21495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:03.175318 21495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:03.175352 21495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:03.175377 21495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:59:03.177003 21495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:03.177214 21495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:03.177464 21495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:03.182489 21504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.182648 21495 server_base.cc:1034] running on GCE node
W20260810 22:59:03.182715 21500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.182525 21502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.182961 21495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:03.183210 21495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:03.184374 21495 hybrid_clock.cc:648] HybridClock initialized: now 1786402743184364 us; error 46 us; skew 500 ppm
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:03.186890 21495 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:59:03.187350 21495 webserver.cc:460] Webserver started at http://127.6.47.129:42163/ using document root <none> and password file <none>
I20260810 22:59:03.187600 21495 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:03.187676 21495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:03.187827 21495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:03.188941 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "cc4b5a93049841ab89fbc604c044b2a0"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "81265a4ab08b8fabf3757b9a7e09afdf"
server_key_iv: "5782dc69a79ba6c8f7e9a70223453ba1"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.189327 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "cc4b5a93049841ab89fbc604c044b2a0"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "81265a4ab08b8fabf3757b9a7e09afdf"
server_key_iv: "5782dc69a79ba6c8f7e9a70223453ba1"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.190668 21495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:59:03.191404 21512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.191579 21495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:03.191699 21495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "cc4b5a93049841ab89fbc604c044b2a0"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "81265a4ab08b8fabf3757b9a7e09afdf"
server_key_iv: "5782dc69a79ba6c8f7e9a70223453ba1"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.191776 21495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.202237 21495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.204074 21495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.204241 21495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.204465 21495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:03.204849 21495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:03.204905 21495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.204941 21495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:03.204968 21495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.211133 21495 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:42033
I20260810 22:59:03.211184 21696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:42033 every 8 connection(s)
I20260810 22:59:03.211491 21495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:59:03.216472 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 21495
I20260810 22:59:03.216619 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:59:03.216815 6334 external_mini_cluster.cc:1428] Setting key ab0c70609aa1a581d95f51b0542385f5
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:03.221747 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.212201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45257 (local address 127.6.47.190:41851)
0810 22:59:03.212250 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.212251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.212476 (+ 225us) server_negotiation.cc:407] Connection header received
0810 22:59:03.213031 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.213032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.213049 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.213101 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.214039 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.214535 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.215553 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.215783 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.215861 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.218376 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:03.218378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:03.218378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:03.218387 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:03.220151 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.220631 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.220632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.220633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.220698 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.220967 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.220968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.220968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.221130 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:03.221193 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:03.221575 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 22:59:03.221686 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 22:59:03.221993 21702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.212156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:45257)
0810 22:59:03.212354 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.212390 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.212923 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.213172 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.213176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.213746 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.213938 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.213942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.214661 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.214662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.215459 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.215461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.215870 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.215871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.215999 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.216574 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.216582 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.218276 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:03.220240 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.220247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.220249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.220550 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.220765 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.220765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.220766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.220902 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.221225 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:03.221233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:03.221392 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 22:59:03.221514 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 22:59:03.221711 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260810 22:59:03.222752 21700 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41851
I20260810 22:59:03.222862 21700 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:03.223058 21700 heartbeater.cc:511] Master 127.6.47.190:41851 requested a full tablet report, sending...
I20260810 22:59:03.223559 21389 ts_manager.cc:194] Registered new tserver with Master: cc4b5a93049841ab89fbc604c044b2a0 (127.6.47.129:42033)
I20260810 22:59:03.224184 21389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:45257
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:59:03.243763 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:03.324456 21705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:03.324723 21705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:03.324746 21705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:03.325872 21705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:03.325907 21705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:03.325933 21705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:59:03.327291 21705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:03.327466 21705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:03.327634 21705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:03.332789 21714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.332816 21711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.333027 21705 server_base.cc:1034] running on GCE node
W20260810 22:59:03.332844 21712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.333284 21705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:03.333531 21705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:03.334678 21705 hybrid_clock.cc:648] HybridClock initialized: now 1786402743334665 us; error 29 us; skew 500 ppm
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:03.337160 21705 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:59:03.337610 21705 webserver.cc:460] Webserver started at http://127.6.47.130:44397/ using document root <none> and password file <none>
I20260810 22:59:03.337798 21705 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:03.337839 21705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:03.337926 21705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:03.339305 21705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "c4e82a6572b8402db25a2f8f789934c1"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "068633f21c4d9dac05f85e0cf912d9d5"
server_key_iv: "f686611b0666232f864518f3df6b961e"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.339635 21705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "c4e82a6572b8402db25a2f8f789934c1"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "068633f21c4d9dac05f85e0cf912d9d5"
server_key_iv: "f686611b0666232f864518f3df6b961e"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.340967 21705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:59:03.341706 21724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.341881 21705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:59:03.341988 21705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "c4e82a6572b8402db25a2f8f789934c1"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "068633f21c4d9dac05f85e0cf912d9d5"
server_key_iv: "f686611b0666232f864518f3df6b961e"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.342064 21705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.350935 21705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.352802 21705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.352964 21705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.353205 21705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:03.353528 21705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:03.353591 21705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.353629 21705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:03.353657 21705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.359611 21705 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40653
I20260810 22:59:03.359654 21906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40653 every 8 connection(s)
I20260810 22:59:03.360004 21705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:03.368736 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 21705
I20260810 22:59:03.368850 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:59:03.368960 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.360835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:56683 (local address 127.6.47.190:41851)
0810 22:59:03.360880 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.360882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.361091 (+ 209us) server_negotiation.cc:407] Connection header received
0810 22:59:03.361558 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.361559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.361570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.361634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.362526 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.363018 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.363961 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.364147 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.364199 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.366267 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:03.366269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:03.366269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:03.366276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:03.367578 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.367957 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.367957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.367958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.368007 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.368220 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.368221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.368221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.368327 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:03.368381 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:03.368745 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 22:59:03.368845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:59:03.369082 6334 external_mini_cluster.cc:1428] Setting key 2cac19d83667b7862fd27426d338f3ff
I20260810 22:59:03.369228 21912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.360792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:56683)
0810 22:59:03.360954 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.360997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.361477 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.361691 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.361694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.362260 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.362451 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.362456 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.363105 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.363105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.363890 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.363892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.364225 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.364225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.364320 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.364821 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.364829 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.366179 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:03.367667 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.367675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.367676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.367892 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.368066 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.368067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.368067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.368166 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.368384 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:03.368389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:03.368592 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 22:59:03.368729 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 22:59:03.368912 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260810 22:59:03.369695 21908 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41851
I20260810 22:59:03.369804 21908 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:03.370000 21908 heartbeater.cc:511] Master 127.6.47.190:41851 requested a full tablet report, sending...
I20260810 22:59:03.370455 21389 ts_manager.cc:194] Registered new tserver with Master: c4e82a6572b8402db25a2f8f789934c1 (127.6.47.130:40653)
I20260810 22:59:03.370833 21389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:56683
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:59:03.397538 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:03.481508 21916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:03.481719 21916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:03.481741 21916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:03.482892 21916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:03.482931 21916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:03.482957 21916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:59:03.484352 21916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:03.484601 21916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:03.484792 21916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:03.489660 21926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.489713 21922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.489782 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.490068 21916 server_base.cc:1034] running on GCE node
I20260810 22:59:03.490253 21916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:03.490486 21916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:03.491636 21916 hybrid_clock.cc:648] HybridClock initialized: now 1786402743491621 us; error 31 us; skew 500 ppm
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:03.494256 21916 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:59:03.494796 21916 webserver.cc:460] Webserver started at http://127.6.47.131:39061/ using document root <none> and password file <none>
I20260810 22:59:03.495052 21916 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:03.495122 21916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:03.495258 21916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:03.496294 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "c723f88ad9c24d7ea0d0b60a87846790"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "dda348c412ca9c3fae0b2c4c69b30db4"
server_key_iv: "d003b0b9457c10b10bf63c99a4a277f3"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.496706 21916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "c723f88ad9c24d7ea0d0b60a87846790"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "dda348c412ca9c3fae0b2c4c69b30db4"
server_key_iv: "d003b0b9457c10b10bf63c99a4a277f3"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.498162 21916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:59:03.498915 21935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.499135 21916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 22:59:03.499244 21916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "c723f88ad9c24d7ea0d0b60a87846790"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "dda348c412ca9c3fae0b2c4c69b30db4"
server_key_iv: "d003b0b9457c10b10bf63c99a4a277f3"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.499317 21916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.513195 21916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.515075 21916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.515249 21916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.515480 21916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:03.515831 21916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:03.515887 21916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.515926 21916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:03.515952 21916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.522722 21916 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:35387
I20260810 22:59:03.522781 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:35387 every 8 connection(s)
I20260810 22:59:03.523103 21916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:59:03.523555 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 21916
I20260810 22:59:03.523701 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:59:03.523917 6334 external_mini_cluster.cc:1428] Setting key f78962ee38e0b615842106664399279e
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:03.532794 21455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.524206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:40789 (local address 127.6.47.190:41851)
0810 22:59:03.524275 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.524277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.524289 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:59:03.524958 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.524960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.524972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.525024 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.525981 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.526486 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.527501 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.527716 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.527807 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.530062 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:03.530063 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:03.530064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:03.530071 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:03.531328 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.531748 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.531750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.531750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.531814 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:03.532081 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:03.532081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:03.532082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:03.532200 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:03.532282 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:03.532616 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 22:59:03.532704 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 22:59:03.532989 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.523848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:40789)
0810 22:59:03.524051 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.524109 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.524840 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.525095 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.525099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.525690 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.525876 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.525881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.526588 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.526589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.527405 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.527407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.527822 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.527823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.527959 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.528419 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.528428 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.529954 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:03.531420 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.531428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.531430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.531665 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.531888 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:03.531889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:03.531889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:03.532002 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:03.532273 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:03.532280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:03.532421 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 22:59:03.532582 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 22:59:03.532748 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260810 22:59:03.533515 22139 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41851
I20260810 22:59:03.533614 22139 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:03.533761 22139 heartbeater.cc:511] Master 127.6.47.190:41851 requested a full tablet report, sending...
I20260810 22:59:03.534135 21389 ts_manager.cc:194] Registered new tserver with Master: c723f88ad9c24d7ea0d0b60a87846790 (127.6.47.131:35387)
I20260810 22:59:03.534461 21389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:40789
I20260810 22:59:03.535673 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
Entry for principal oryx/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab.
WARNING: no policy specified for oryx/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
Entry for principal oryx/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab.
WARNING: no policy specified for oryx/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
Entry for principal oryx/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab.
I20260810 22:59:03.608564 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 21495
I20260810 22:59:03.622961 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 21705
I20260810 22:59:03.643217 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 21916
I20260810 22:59:03.663337 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 21337
I20260810 22:59:03.676659 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41851
--webserver_interface=127.6.47.190
--webserver_port=35225
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:03.762423 22154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:03.762624 22154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:03.762646 22154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:03.763818 22154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:03.763863 22154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:03.763878 22154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:03.763891 22154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:03.763916 22154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:03.765407 22154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=35225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:03.765671 22154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:03.765905 22154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:03.770910 22163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.770913 22154 server_base.cc:1034] running on GCE node
W20260810 22:59:03.770942 22169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.770933 22165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.771406 22154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:03.771759 22154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:03.772922 22154 hybrid_clock.cc:648] HybridClock initialized: now 1786402743772917 us; error 36 us; skew 500 ppm
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:03.775339 22154 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:59:03.775771 22154 webserver.cc:460] Webserver started at http://127.6.47.190:35225/ using document root <none> and password file <none>
I20260810 22:59:03.776013 22154 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:03.776096 22154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:03.777510 22154 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:03.778257 22187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.778430 22154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:03.778541 22154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "f50edd59ca6f4347811adc1f6ab918f6"
format_stamp: "Formatted at 2026-08-10 22:59:02 on dist-test-slave-rwrg"
server_key: "f8cc2649ced70f4f20e3210aa3d334ea"
server_key_iv: "827016379992f14a3a9ce52a55b67bce"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.778898 22154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.801931 22154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.803956 22154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.804131 22154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.808365 22154 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:41851
I20260810 22:59:03.808401 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:41851 every 8 connection(s)
I20260810 22:59:03.808826 22154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:59:03.810348 22279 sys_catalog.cc:263] Verifying existing consensus state
I20260810 22:59:03.811055 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Bootstrap starting.
I20260810 22:59:03.817906 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 22154
I20260810 22:59:03.818228 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:42033
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:41851
--webserver_port=42163
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 22:59:03.819749 22279 log.cc:824] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:03.821556 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 22:59:03.821857 22279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Bootstrap complete.
I20260810 22:59:03.824122 22279 raft_consensus.cc:348] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.824380 22279 raft_consensus.cc:729] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50edd59ca6f4347811adc1f6ab918f6, State: Initialized, Role: FOLLOWER
I20260810 22:59:03.824599 22279 consensus_queue.cc:260] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.824688 22279 raft_consensus.cc:388] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:59:03.824743 22279 raft_consensus.cc:482] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:59:03.824790 22279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 1 FOLLOWER]: Advancing to term 2
I20260810 22:59:03.825773 22279 raft_consensus.cc:504] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.825893 22279 leader_election.cc:302] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f50edd59ca6f4347811adc1f6ab918f6; no voters:
I20260810 22:59:03.826087 22279 leader_election.cc:288] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 22:59:03.826126 22287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 22:59:03.826254 22287 raft_consensus.cc:686] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [term 2 LEADER]: Becoming Leader. State: Replica: f50edd59ca6f4347811adc1f6ab918f6, State: Running, Role: LEADER
I20260810 22:59:03.826360 22279 sys_catalog.cc:564] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:59:03.826411 22287 consensus_queue.cc:237] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } }
I20260810 22:59:03.826864 22289 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f50edd59ca6f4347811adc1f6ab918f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } } }
I20260810 22:59:03.826958 22289 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:03.827391 22297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:59:03.827368 22290 sys_catalog.cc:455] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f50edd59ca6f4347811adc1f6ab918f6. Latest consensus state: current_term: 2 leader_uuid: "f50edd59ca6f4347811adc1f6ab918f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f50edd59ca6f4347811adc1f6ab918f6" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41851 } } }
I20260810 22:59:03.827523 22290 sys_catalog.cc:458] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:03.828001 22297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:59:03.828400 22297 catalog_manager.cc:1259] Loaded cluster ID: 2a6e55ef802d425bac470c376dced5dc
I20260810 22:59:03.828451 22297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:59:03.829866 22297 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:59:03.830032 22297 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f50edd59ca6f4347811adc1f6ab918f6: Loaded TSK: 0
I20260810 22:59:03.830250 22297 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 22:59:03.904938 22283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:03.905253 22283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:03.905295 22283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:03.907042 22283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:03.907133 22283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:03.907179 22283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:59:03.908741 22283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.129.keytab
--principal=oryx/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:42033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=42163
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:03.909013 22283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:03.909291 22283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:03.915378 22324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.915354 22321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:03.915359 22319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:03.915877 22283 server_base.cc:1034] running on GCE node
I20260810 22:59:03.916077 22283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:03.916349 22283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:03.917526 22283 hybrid_clock.cc:648] HybridClock initialized: now 1786402743917488 us; error 51 us; skew 500 ppm
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402743, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:03.920743 22283 init.cc:377] Logged in from keytab as oryx/127.6.47.129@KRBTEST.COM (short username oryx)
I20260810 22:59:03.921185 22283 webserver.cc:460] Webserver started at http://127.6.47.129:42163/ using document root <none> and password file <none>
I20260810 22:59:03.921465 22283 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:03.921552 22283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:03.922991 22283 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:59:03.923660 22339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.923851 22283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 22:59:03.923949 22283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "cc4b5a93049841ab89fbc604c044b2a0"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "81265a4ab08b8fabf3757b9a7e09afdf"
server_key_iv: "5782dc69a79ba6c8f7e9a70223453ba1"
server_key_version: "encryptionkey@0"
I20260810 22:59:03.924321 22283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:03.953738 22283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:03.955760 22283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:03.955936 22283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:03.956207 22283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:03.956604 22283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:03.956658 22283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.956700 22283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:03.956727 22283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:03.962515 22283 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:42033
I20260810 22:59:03.962560 22551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:42033 every 8 connection(s)
I20260810 22:59:03.962970 22283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:59:03.964543 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 22283
I20260810 22:59:03.964882 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:40653
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:41851
--webserver_port=44397
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:03.973276 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.963735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:51143)
0810 22:59:03.963927 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.963971 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.964595 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.965250 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.965254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.966078 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.966345 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.966351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.967916 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.967918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.969336 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.969340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.969918 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.969920 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.970086 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.970626 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.970637 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.973129 (+ 2492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":108,"threads_started":1}
W20260810 22:59:03.973543 22555 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:41851 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.47.190:41851: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
W20260810 22:59:03.973928 22561 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:03.963812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:51143 (local address 127.6.47.190:41851)
0810 22:59:03.963999 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.964007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.964041 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:59:03.964913 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.964920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.964980 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.965104 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.966527 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.967803 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.969432 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.969822 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.969906 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.973320 (+ 3414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:51143: BlockingRecv error: recv got EOF from 127.6.47.129:51143 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:03.978686 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.973792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:42697)
0810 22:59:03.974008 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.974020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.974072 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.974262 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.974263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.974421 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.974484 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.974486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.975112 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.975114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.975987 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.975991 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.976338 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.976340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.976417 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.976797 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.976803 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.978621 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195}
W20260810 22:59:03.978778 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:03.973855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:42697 (local address 127.6.47.190:41851)
0810 22:59:03.974020 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.974023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.974123 (+ 100us) server_negotiation.cc:407] Connection header received
0810 22:59:03.974146 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.974147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.974158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.974221 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.974566 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.975031 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.976080 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.976261 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.976313 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.978709 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:42697: BlockingRecv error: recv got EOF from 127.6.47.129:42697 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:03 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:03.983409 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:03.979030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:56581)
0810 22:59:03.979070 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:03.979079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:03.979111 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:03.979273 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:03.979273 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:03.979406 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:03.979438 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.979439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.979980 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.979981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.980776 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:03.980778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.981144 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.981145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:03.981204 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.981612 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:03.981616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:03.983346 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260810 22:59:03.983464 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:03.979094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:56581 (local address 127.6.47.190:41851)
0810 22:59:03.979162 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:03.979163 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:03.979177 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:59:03.979195 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:03.979195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:03.979203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:03.979249 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:03.979502 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.979926 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.980853 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:03.981070 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:03.981129 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:03.983427 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:56581: BlockingRecv error: recv got EOF from 127.6.47.129:56581 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 22:59:03.983771 22555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:59:04.053236 22562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:04.053469 22562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:04.053509 22562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:04.054680 22562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:04.054764 22562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:04.054811 22562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:59:04.056285 22562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.130.keytab
--principal=oryx/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:40653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=44397
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:04.056581 22562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:04.056816 22562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:04.061837 22575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:04.061823 22572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:04.061823 22581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:04.062093 22562 server_base.cc:1034] running on GCE node
I20260810 22:59:04.062312 22562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:04.062561 22562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:04.063714 22562 hybrid_clock.cc:648] HybridClock initialized: now 1786402744063696 us; error 39 us; skew 500 ppm
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402744, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:04.066293 22562 init.cc:377] Logged in from keytab as oryx/127.6.47.130@KRBTEST.COM (short username oryx)
I20260810 22:59:04.066751 22562 webserver.cc:460] Webserver started at http://127.6.47.130:44397/ using document root <none> and password file <none>
I20260810 22:59:04.067005 22562 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:04.067103 22562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:04.068482 22562 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:04.069155 22594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.069339 22562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:59:04.069448 22562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "c4e82a6572b8402db25a2f8f789934c1"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "068633f21c4d9dac05f85e0cf912d9d5"
server_key_iv: "f686611b0666232f864518f3df6b961e"
server_key_version: "encryptionkey@0"
I20260810 22:59:04.069818 22562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:04.093518 22562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:04.098769 22562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:04.098946 22562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:04.099244 22562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:04.099586 22562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:04.099648 22562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.099695 22562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:04.099715 22562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.106796 22562 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40653
I20260810 22:59:04.106900 22789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40653 every 8 connection(s)
I20260810 22:59:04.107334 22562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:59:04.111866 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 22562
I20260810 22:59:04.112182 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:35387
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:41851
--webserver_port=39061
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:59:04.117197 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.108259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:49007 (local address 127.6.47.190:41851)
0810 22:59:04.108305 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.108307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.108680 (+ 373us) server_negotiation.cc:407] Connection header received
0810 22:59:04.109185 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.109187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.109203 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.109282 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.110575 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.111506 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.113158 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.113487 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.113566 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.117121 (+ 3555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:49007: BlockingRecv error: recv got EOF from 127.6.47.130:49007 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:59:04.117350 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.108280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:49007)
0810 22:59:04.108483 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.108555 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.109072 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.109375 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.109379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.110101 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.110427 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.110433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.111893 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.111894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.113061 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.113064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.113586 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.113587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.113766 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.114413 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.114425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.117018 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":94,"threads_started":1}
W20260810 22:59:04.117602 22792 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:41851 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.47.190:41851: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:04.123744 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.117840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:35619)
0810 22:59:04.117891 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.117903 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.117953 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.118090 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.118091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.118261 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.118330 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.118332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.119325 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.119326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.120244 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.120259 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.120853 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.120854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.120932 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.121437 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.121443 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.123651 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 22:59:04.123867 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.117892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:35619 (local address 127.6.47.190:41851)
0810 22:59:04.117932 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.117934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.117944 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:59:04.118009 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.118010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.118023 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.118086 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.118412 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.119228 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.120353 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.120751 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.120824 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.123784 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:35619: BlockingRecv error: recv got EOF from 127.6.47.130:35619 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:59:04.129870 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.124398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:57907 (local address 127.6.47.190:41851)
0810 22:59:04.124435 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.124437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.124447 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:59:04.124475 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.124475 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.124486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.124568 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.124915 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.125667 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.126855 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.127204 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.127271 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.129811 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:57907: BlockingRecv error: recv got EOF from 127.6.47.130:57907 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:59:04.130036 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.124330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:57907)
0810 22:59:04.124378 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.124388 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.124425 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.124623 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.124624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.124789 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.124825 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.124827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.125813 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.125814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.126696 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.126699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.127274 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.127275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.127375 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.127789 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.127794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.129978 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 22:59:04.130252 22792 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 22:59:04.201485 22795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:04.201699 22795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:04.201723 22795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:04.202898 22795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:04.202942 22795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:04.202996 22795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:59:04.204476 22795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/krb5kdc/oryx_127.6.47.131.keytab
--principal=oryx/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:35387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=39061
--tserver_master_addrs=127.6.47.190:41851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:04.204793 22795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:04.205035 22795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:04.210140 22807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:04.210160 22809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:04.210141 22812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:04.210502 22795 server_base.cc:1034] running on GCE node
I20260810 22:59:04.210706 22795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:04.210908 22795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:04.212072 22795 hybrid_clock.cc:648] HybridClock initialized: now 1786402744212077 us; error 45 us; skew 500 ppm
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402744, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:04.214635 22795 init.cc:377] Logged in from keytab as oryx/127.6.47.131@KRBTEST.COM (short username oryx)
I20260810 22:59:04.215126 22795 webserver.cc:460] Webserver started at http://127.6.47.131:39061/ using document root <none> and password file <none>
I20260810 22:59:04.215371 22795 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:04.215456 22795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:04.216873 22795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 22:59:04.217516 22825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.217695 22795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:59:04.217803 22795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "c723f88ad9c24d7ea0d0b60a87846790"
format_stamp: "Formatted at 2026-08-10 22:59:03 on dist-test-slave-rwrg"
server_key: "dda348c412ca9c3fae0b2c4c69b30db4"
server_key_iv: "d003b0b9457c10b10bf63c99a4a277f3"
server_key_version: "encryptionkey@0"
I20260810 22:59:04.218151 22795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:04.241624 22795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:04.243714 22795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:04.243883 22795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:04.244140 22795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:04.244458 22795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:04.244557 22795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.244608 22795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:04.244637 22795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:04.250581 22795 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:35387
I20260810 22:59:04.250684 22995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:35387 every 8 connection(s)
I20260810 22:59:04.251003 22795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 22:59:04.257906 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.251787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:59703 (local address 127.6.47.190:41851)
0810 22:59:04.251833 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.251835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.252030 (+ 195us) server_negotiation.cc:407] Connection header received
0810 22:59:04.252531 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.252532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.252560 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.252611 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.253538 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.254012 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.254931 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.255110 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.255159 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.257848 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:59703: BlockingRecv error: recv got EOF from 127.6.47.131:59703 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 22:59:04.257972 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.251753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:59703)
0810 22:59:04.251913 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.251954 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.252430 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.252696 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.252700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.253269 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.253456 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.253460 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.254096 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.254096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.254856 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.254857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.255189 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.255190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.255285 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.255681 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.255689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.257784 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
W20260810 22:59:04.258208 22996 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:41851 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.47.190:41851: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
I20260810 22:59:04.259151 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 22795
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:04.263059 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.258423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:53233)
0810 22:59:04.258471 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.258482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.258529 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.258768 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.258769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.258889 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.258930 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.258931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.259667 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.259668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.260237 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.260239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.260712 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.260714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.260778 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.261139 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.261144 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.263006 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 22:59:04.263163 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.258603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:53233 (local address 127.6.47.190:41851)
0810 22:59:04.258657 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.258659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.258665 (+ 6us) server_negotiation.cc:407] Connection header received
0810 22:59:04.258682 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.258682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.258689 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.258746 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.259033 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.259579 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.260321 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.260628 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.260683 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.263102 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:53233: BlockingRecv error: recv got EOF from 127.6.47.131:53233 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:04.267429 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.259537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.6.47.190:41851)
0810 22:59:04.259594 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.259596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.259748 (+ 152us) server_negotiation.cc:407] Connection header received
0810 22:59:04.259786 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.259786 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.259799 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.259854 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.260439 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.261157 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.262121 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.262416 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.262467 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.264133 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:04.264137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:04.264137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:04.264148 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:04.265891 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:04.266293 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:04.266296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:04.266296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:04.266385 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:04.266660 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:04.266662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:04.266662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:04.266915 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:04.267060 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:04.267201 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 22:59:04.267318 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:59:04.267620 22998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.263317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:36305)
0810 22:59:04.263370 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.263381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.263419 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.263543 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.263544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.263695 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.263732 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.263734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.264305 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.264306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.265020 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.265023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.265455 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.265456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.265492 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.265840 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.265845 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.267542 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 22:59:04.267637 22563 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.263378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:36305 (local address 127.6.47.190:41851)
0810 22:59:04.263413 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.263415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.263424 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:04.263469 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.263470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.263477 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.263523 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.263806 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.264216 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.265102 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.265377 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.265454 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.267596 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:36305: BlockingRecv error: recv got EOF from 127.6.47.131:36305 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 22:59:04.267836 22996 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:04 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:04.989193 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:04.984272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:60339)
0810 22:59:04.984413 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:04.984423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:04.984471 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:04.984689 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:04.984690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:04.984841 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:04.984908 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.984909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.985755 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.985756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.986280 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:04.986282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.986656 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.986657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:04.986726 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.987155 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:04.987160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:04.989131 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260810 22:59:04.989363 23003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:04.984335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:60339 (local address 127.6.47.190:41851)
0810 22:59:04.984464 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:04.984466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:04.984474 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:59:04.984586 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:04.984587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:04.984597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:04.984663 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:04.984989 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.985678 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.986364 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:04.986581 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:04.986657 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:04.989274 (+ 2617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:60339: BlockingRecv error: recv got EOF from 127.6.47.129:60339 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:05.135180 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:05.130657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:38297)
0810 22:59:05.130841 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:05.130850 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:05.130905 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:05.131020 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:05.131020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:05.131188 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:05.131237 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.131239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.131840 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.131841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.132408 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.132410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.132794 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.132795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.132839 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.133290 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:05.133295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:05.135107 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
W20260810 22:59:05.135277 23003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:05.130738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:38297 (local address 127.6.47.190:41851)
0810 22:59:05.130816 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:05.130818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:05.130907 (+ 89us) server_negotiation.cc:407] Connection header received
0810 22:59:05.130949 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:05.130950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:05.130961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:05.131006 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:05.131298 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.131772 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.132484 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.132724 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.132783 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.135211 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:38297: BlockingRecv error: recv got EOF from 127.6.47.130:38297 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:05.272928 23005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:05.268195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:34829)
0810 22:59:05.268370 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:05.268382 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:05.268468 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:05.268686 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:05.268688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:05.268864 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:05.268913 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.268915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.269605 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.269607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.270161 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.270163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.270520 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.270521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.270579 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.271084 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:05.271090 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:05.272862 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
W20260810 22:59:05.273006 23003 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:05.268277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:34829 (local address 127.6.47.190:41851)
0810 22:59:05.268353 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:05.268355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:05.268448 (+ 93us) server_negotiation.cc:407] Connection header received
0810 22:59:05.268577 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:05.268578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:05.268593 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:05.268666 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:05.268994 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.269508 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.270246 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.270443 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.270522 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.272956 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:34829: BlockingRecv error: recv got EOF from 127.6.47.131:34829 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:05 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:05.994347 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:05.989701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:40741)
0810 22:59:05.989850 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:05.989861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:05.989910 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:05.990082 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:05.990083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:05.990253 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:05.990303 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.990305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.990957 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.990958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.991479 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:05.991481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.991821 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.991822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:05.991863 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.992390 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:05.992395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:05.994244 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260810 22:59:05.994540 23009 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:05.989788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:40741 (local address 127.6.47.190:41851)
0810 22:59:05.989937 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:05.989939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:05.989949 (+ 10us) server_negotiation.cc:407] Connection header received
0810 22:59:05.989983 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:05.989984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:05.989994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:05.990062 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:05.990370 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.990874 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.991543 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:05.991753 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:05.991807 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:05.994366 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:40741: BlockingRecv error: recv got EOF from 127.6.47.129:40741 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:06.140177 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:06.135802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:55721)
0810 22:59:06.135962 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:06.135972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:06.136030 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:06.136156 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:06.136158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:06.136321 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:06.136370 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.136371 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.137002 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.137003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.137505 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.137506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.137853 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.137854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.137893 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:06.138323 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:06.138328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:06.140114 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
W20260810 22:59:06.140322 23009 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:06.135828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:55721 (local address 127.6.47.190:41851)
0810 22:59:06.135896 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:06.135898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:06.136034 (+ 136us) server_negotiation.cc:407] Connection header received
0810 22:59:06.136072 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:06.136073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:06.136087 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:06.136139 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:06.136433 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.136936 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.137578 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.137786 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.137839 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:06.140236 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:55721: BlockingRecv error: recv got EOF from 127.6.47.130:55721 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:06.278261 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:06.273573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:44507)
0810 22:59:06.273704 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:06.273715 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:06.273775 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:06.273944 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:06.273945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:06.274142 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:06.274191 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.274193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.274846 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.274847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.275424 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.275427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.275801 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.275803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.275864 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:06.276384 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:06.276389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:06.278190 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
W20260810 22:59:06.278389 23009 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:06.273739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:44507 (local address 127.6.47.190:41851)
0810 22:59:06.273813 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:06.273815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:06.273824 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:06.273856 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:06.273857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:06.273869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:06.273933 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:06.274274 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.274760 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.275526 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.275722 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.275800 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:06.278286 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:44507: BlockingRecv error: recv got EOF from 127.6.47.131:44507 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:06 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:07.002530 23013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:06.994786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:42195)
0810 22:59:06.994945 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:06.994957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:06.995010 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:06.995145 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:06.995146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:06.995304 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:06.995355 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.995356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.995987 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.995988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.996483 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:06.996485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.996859 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.996860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:06.996899 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.000594 (+ 3695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:07.000600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:07.002471 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
W20260810 22:59:07.002705 23015 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:06.994876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:42195 (local address 127.6.47.190:41851)
0810 22:59:06.994993 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:06.994995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:06.995004 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:06.995066 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:06.995067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:06.995076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:06.995129 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:06.995423 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.995921 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.996585 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:06.996788 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:06.996843 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.002601 (+ 5758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:42195: BlockingRecv error: recv got EOF from 127.6.47.129:42195 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
Aug 10 22:59:07 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:07 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:07.145181 23016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:07.140687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:53297)
0810 22:59:07.140858 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:07.140868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:07.140932 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:07.141067 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:07.141068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:07.141267 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:07.141317 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:07.141318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.141937 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.141938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:07.142485 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:07.142487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.142823 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.142824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:07.142864 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.143320 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:07.143324 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:07.145100 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
W20260810 22:59:07.145310 23015 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:07.140750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:53297 (local address 127.6.47.190:41851)
0810 22:59:07.140805 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:07.140807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:07.140936 (+ 129us) server_negotiation.cc:407] Connection header received
0810 22:59:07.140985 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:07.140986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:07.140996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:07.141049 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:07.141380 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.141867 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.142559 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.142752 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.142803 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.145207 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:53297: BlockingRecv error: recv got EOF from 127.6.47.130:53297 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 22:59:07 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:07 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:07.283602 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:07.278794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:53113)
0810 22:59:07.278923 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:07.278933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:07.278987 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:07.279129 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:07.279129 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:07.279303 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:07.279353 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:07.279354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.279957 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.279958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:07.280460 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:07.280461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.280923 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.280924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:07.280990 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.281565 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:07.281571 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:07.283505 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
W20260810 22:59:07.283664 23015 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:07.278879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:53113 (local address 127.6.47.190:41851)
0810 22:59:07.278923 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:07.278925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:07.278974 (+ 49us) server_negotiation.cc:407] Connection header received
0810 22:59:07.279046 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:07.279046 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:07.279057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:07.279098 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:07.279421 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.279891 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.280607 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:07.280777 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:07.280914 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:07.283599 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:53113: BlockingRecv error: recv got EOF from 127.6.47.131:53113 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:08.007804 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:08.003033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:49909)
0810 22:59:08.003197 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:08.003210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:08.003284 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:08.003470 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:08.003471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:08.003676 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:08.003728 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.003730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.004374 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.004375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.005107 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.005111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.005455 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.005455 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.005496 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.005984 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:08.005988 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:08.007746 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
W20260810 22:59:08.007925 23021 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:08.003102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:49909 (local address 127.6.47.190:41851)
0810 22:59:08.003229 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:08.003231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:08.003282 (+ 51us) server_negotiation.cc:407] Connection header received
0810 22:59:08.003348 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:08.003349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:08.003360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:08.003411 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:08.003796 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.004292 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.005188 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.005382 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.005437 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.007843 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:49909: BlockingRecv error: recv got EOF from 127.6.47.129:49909 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":83,"threads_started":1}
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:08.150569 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:08.145685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:34723)
0810 22:59:08.145904 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:08.145917 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:08.146007 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:08.146182 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:08.146183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:08.146371 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:08.146417 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.146419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.147111 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.147112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.147678 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.147680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.148050 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.148051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.148110 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.148678 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:08.148683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:08.150472 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":125,"threads_started":1}
W20260810 22:59:08.150624 23021 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:08.145782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:34723 (local address 127.6.47.190:41851)
0810 22:59:08.145889 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:08.145892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:08.145992 (+ 100us) server_negotiation.cc:407] Connection header received
0810 22:59:08.146067 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:08.146068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:08.146082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:08.146161 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:08.146507 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.147011 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.147770 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.147968 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.148052 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.150558 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:34723: BlockingRecv error: recv got EOF from 127.6.47.130:34723 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:08 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:08.289186 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:08.284225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:39557)
0810 22:59:08.284404 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:08.284415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:08.284474 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:08.284692 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:08.284693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:08.284930 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:08.284981 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.284984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.285867 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.285869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.286400 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:08.286401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.286753 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.286754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:08.286795 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.287249 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:08.287254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:08.289126 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
W20260810 22:59:08.289286 23021 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:08.284319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:39557 (local address 127.6.47.190:41851)
0810 22:59:08.284394 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:08.284396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:08.284481 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:59:08.284606 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:08.284607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:08.284620 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:08.284692 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:08.285068 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.285789 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.286496 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:08.286680 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:08.286731 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:08.289223 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:39557: BlockingRecv error: recv got EOF from 127.6.47.131:39557 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:09.013006 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:09.008397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:56905)
0810 22:59:09.008576 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:09.008586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:09.008637 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:09.008796 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:09.008797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:09.008963 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:09.009013 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.009014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.009685 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.009686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.010190 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.010191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.010518 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.010519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.010559 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.011030 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:09.011034 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:09.012926 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
W20260810 22:59:09.013183 23026 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:09.008474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:56905 (local address 127.6.47.190:41851)
0810 22:59:09.008653 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:09.008655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:09.008666 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:59:09.008698 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:09.008699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:09.008709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:09.008772 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:09.009086 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.009606 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.010266 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.010449 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.010501 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.013073 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:56905: BlockingRecv error: recv got EOF from 127.6.47.129:56905 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:09.155884 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:09.151206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:41105)
0810 22:59:09.151415 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:09.151425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:09.151482 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:09.151659 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:09.151660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:09.151834 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:09.151883 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.151885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.152661 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.152662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.153190 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.153192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.153525 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.153526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.153565 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.154023 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:09.154027 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:09.155825 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":112,"threads_started":1}
W20260810 22:59:09.156024 23026 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:09.151327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:41105 (local address 127.6.47.190:41851)
0810 22:59:09.151413 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:09.151415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:09.151490 (+ 75us) server_negotiation.cc:407] Connection header received
0810 22:59:09.151531 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:09.151532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:09.151541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:09.151624 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:09.151951 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.152485 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.153270 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.153457 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.153509 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.155942 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:41105: BlockingRecv error: recv got EOF from 127.6.47.130:41105 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:09 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:09.294237 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:09.289771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:52857)
0810 22:59:09.289957 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:09.289967 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:09.290027 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:09.290134 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:09.290134 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:09.290295 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:09.290345 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.290346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.291008 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.291008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.291522 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:09.291523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.291847 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.291848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:09.291888 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.292335 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:09.292339 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:09.294177 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
W20260810 22:59:09.294345 23026 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:09.289816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:52857 (local address 127.6.47.190:41851)
0810 22:59:09.289892 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:09.289895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:09.290034 (+ 139us) server_negotiation.cc:407] Connection header received
0810 22:59:09.290066 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:09.290067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:09.290079 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:09.290128 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:09.290407 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.290935 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.291601 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:09.291779 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:09.291830 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:09.294297 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:52857: BlockingRecv error: recv got EOF from 127.6.47.131:52857 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:10.018133 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:10.013520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:33013)
0810 22:59:10.013687 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:10.013699 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:10.013753 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:10.013911 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:10.013912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:10.014070 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:10.014122 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.014123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.014827 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.014829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.015384 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.015385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.015761 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.015762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.015804 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.016283 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:10.016287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:10.018030 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
W20260810 22:59:10.018262 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:10.013610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:33013 (local address 127.6.47.190:41851)
0810 22:59:10.013771 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:10.013773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:10.013784 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:59:10.013808 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:10.013809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:10.013819 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:10.013890 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:10.014202 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.014698 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.015476 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.015677 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.015736 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.018144 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:33013: BlockingRecv error: recv got EOF from 127.6.47.129:33013 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:10.161568 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:10.156558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:56387)
0810 22:59:10.156751 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:10.156764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:10.156825 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:10.156980 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:10.156981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:10.157170 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:10.157219 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.157221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.157851 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.157852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.158464 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.158467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.158806 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.158807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.158859 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.159434 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:10.159439 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:10.161509 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":133,"threads_started":1}
W20260810 22:59:10.161715 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:10.156746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:56387 (local address 127.6.47.190:41851)
0810 22:59:10.156797 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:10.156799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:10.156816 (+ 17us) server_negotiation.cc:407] Connection header received
0810 22:59:10.156881 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:10.156882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:10.156893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:10.156956 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:10.157296 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.157781 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.158543 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.158735 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.158793 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.161641 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:56387: BlockingRecv error: recv got EOF from 127.6.47.130:56387 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:10 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:10.299422 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:10.294921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:54671)
0810 22:59:10.295111 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:10.295123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:10.295190 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:10.295321 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:10.295321 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:10.295495 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:10.295547 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.295548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.296153 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.296154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.296739 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:10.296742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.297081 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.297081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:10.297117 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.297580 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:10.297584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:10.299357 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":123,"threads_started":1}
W20260810 22:59:10.299561 23032 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:10.294980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:54671 (local address 127.6.47.190:41851)
0810 22:59:10.295052 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:10.295054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:10.295197 (+ 143us) server_negotiation.cc:407] Connection header received
0810 22:59:10.295237 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:10.295238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:10.295250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:10.295299 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:10.295613 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.296088 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.296826 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:10.297003 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:10.297069 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:10.299465 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:54671: BlockingRecv error: recv got EOF from 127.6.47.131:54671 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:11.024128 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:11.018768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:36815)
0810 22:59:11.018946 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:11.018958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:11.019016 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:11.019206 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:11.019207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:11.019369 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:11.019420 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.019422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.020080 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.020080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.020732 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.020735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.021135 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.021136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.021185 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.021770 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:11.021774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:11.024000 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":107,"threads_started":1}
W20260810 22:59:11.024168 23039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:11.018894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:36815 (local address 127.6.47.190:41851)
0810 22:59:11.019063 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:11.019065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:11.019079 (+ 14us) server_negotiation.cc:407] Connection header received
0810 22:59:11.019109 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:11.019110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:11.019121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:11.019181 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:11.019496 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.020008 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.020818 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.021063 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.021123 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.024088 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:36815: BlockingRecv error: recv got EOF from 127.6.47.129:36815 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:11.167228 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:11.162091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:50097)
0810 22:59:11.162293 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:11.162307 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:11.162380 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:11.162564 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:11.162565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:11.162744 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:11.162795 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.162797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.163749 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.163751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.164324 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.164327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.164720 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.164721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.164768 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.165208 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:11.165212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:11.167163 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":140,"threads_started":1}
W20260810 22:59:11.167382 23039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:11.162204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:50097 (local address 127.6.47.190:41851)
0810 22:59:11.162290 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:11.162292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:11.162379 (+ 87us) server_negotiation.cc:407] Connection header received
0810 22:59:11.162428 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:11.162429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:11.162440 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:11.162497 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:11.162866 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.163623 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.164414 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.164648 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.164708 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.167285 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:50097: BlockingRecv error: recv got EOF from 127.6.47.130:50097 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:11 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:11.305169 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:11.300233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:44389)
0810 22:59:11.300435 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:11.300451 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:11.300573 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:11.300773 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:11.300774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:11.300957 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:11.301010 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.301011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.301663 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.301664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.302198 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:11.302200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.302548 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.302549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:11.302590 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.303107 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:11.303111 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:11.305101 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":134,"threads_started":1}
W20260810 22:59:11.305243 23039 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:11.300383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:44389 (local address 127.6.47.190:41851)
0810 22:59:11.300464 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:11.300467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:11.300511 (+ 44us) server_negotiation.cc:407] Connection header received
0810 22:59:11.300667 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:11.300668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:11.300681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:11.300743 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:11.301094 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.301586 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.302284 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:11.302471 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:11.302536 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:11.305193 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:44389: BlockingRecv error: recv got EOF from 127.6.47.131:44389 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:12.033279 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:12.024687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:45043)
0810 22:59:12.024877 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:12.024895 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:12.024974 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:12.025144 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:12.025145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:12.025355 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:12.025414 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.025416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.026070 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.026071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.026755 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.026759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.027150 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.027151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.027193 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.031166 (+ 3973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:12.031171 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:12.033176 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
W20260810 22:59:12.033341 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:12.024855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45043 (local address 127.6.47.190:41851)
0810 22:59:12.024998 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:12.025000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:12.025009 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:12.025054 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:12.025055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:12.025064 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:12.025122 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:12.025481 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.025983 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.026829 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.027078 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.027140 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.033263 (+ 6123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:45043: BlockingRecv error: recv got EOF from 127.6.47.129:45043 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:12.172726 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:12.167746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:33185)
0810 22:59:12.167888 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:12.167898 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:12.167966 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:12.168128 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:12.168129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:12.168309 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:12.168356 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.168358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.169256 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.169257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.169797 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.169799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.170160 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.170161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.170225 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.170664 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:12.170669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:12.172612 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260810 22:59:12.172770 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:12.167914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:33185 (local address 127.6.47.190:41851)
0810 22:59:12.167991 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:12.167993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:12.168004 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:59:12.168035 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:12.168036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:12.168051 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:12.168110 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:12.168448 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.169150 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.169882 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.170081 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.170159 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.172702 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:33185: BlockingRecv error: recv got EOF from 127.6.47.130:33185 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:12 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:12.310631 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:12.305833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:49147)
0810 22:59:12.306025 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:12.306036 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:12.306135 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:12.306318 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:12.306319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:12.306506 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:12.306556 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.306558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.307225 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.307227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.307765 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:12.307767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.308127 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.308129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:12.308185 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.308780 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:12.308785 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:12.310569 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
W20260810 22:59:12.310778 23045 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:12.305928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:49147 (local address 127.6.47.190:41851)
0810 22:59:12.306003 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:12.306006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:12.306104 (+ 98us) server_negotiation.cc:407] Connection header received
0810 22:59:12.306223 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:12.306224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:12.306239 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:12.306295 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:12.306642 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.307141 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.307849 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:12.308039 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:12.308128 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:12.310694 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:49147: BlockingRecv error: recv got EOF from 127.6.47.131:49147 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:13.038748 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:13.033841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:48527)
0810 22:59:13.034004 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:13.034018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:13.034086 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:13.034234 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:13.034234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:13.034406 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:13.034459 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.034461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.035105 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.035105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.035622 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.035624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.035940 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.035941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.035980 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.036486 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:13.036530 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:13.038635 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260810 22:59:13.038794 23054 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:13.033905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:48527 (local address 127.6.47.190:41851)
0810 22:59:13.034056 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:13.034058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:13.034071 (+ 13us) server_negotiation.cc:407] Connection header received
0810 22:59:13.034143 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:13.034144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:13.034154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:13.034215 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:13.034520 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.035039 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.035690 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.035878 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.035930 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.038719 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:48527: BlockingRecv error: recv got EOF from 127.6.47.129:48527 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:13.177706 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:13.173238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:49583)
0810 22:59:13.173408 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:13.173420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:13.173485 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:13.173612 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:13.173613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:13.173819 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:13.173869 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.173871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.174498 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.174499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.175014 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.175015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.175341 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.175342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.175383 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.175849 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:13.175853 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:13.177650 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260810 22:59:13.177817 23054 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:13.173299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:49583 (local address 127.6.47.190:41851)
0810 22:59:13.173353 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:13.173356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:13.173485 (+ 129us) server_negotiation.cc:407] Connection header received
0810 22:59:13.173534 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:13.173535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:13.173546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:13.173609 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:13.173933 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.174431 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.175083 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.175277 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.175329 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.177752 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:49583: BlockingRecv error: recv got EOF from 127.6.47.130:49583 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:13 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:13.315734 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:13.311088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:48619)
0810 22:59:13.311225 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:13.311240 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:13.311303 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:13.311472 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:13.311473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:13.311672 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:13.311723 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.311725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.312372 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.312373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.312940 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:13.312942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.313286 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.313287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:13.313327 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.313779 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:13.313784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:13.315652 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
W20260810 22:59:13.315828 23054 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:13.311260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:48619 (local address 127.6.47.190:41851)
0810 22:59:13.311354 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:13.311356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:13.311364 (+ 8us) server_negotiation.cc:407] Connection header received
0810 22:59:13.311387 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:13.311388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:13.311400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:13.311457 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:13.311790 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.312276 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.313011 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:13.313221 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:13.313276 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:13.315743 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:48619: BlockingRecv error: recv got EOF from 127.6.47.131:48619 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:14.049204 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:14.039228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:50097)
0810 22:59:14.039390 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:14.039407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:14.039464 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:14.039603 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:14.039604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:14.039776 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:14.039828 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.039830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.040470 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.040471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.041066 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.041068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.041395 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.041397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.041438 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.047189 (+ 5751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:14.047199 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:14.049091 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
W20260810 22:59:14.049286 23059 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:14.039286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50097 (local address 127.6.47.190:41851)
0810 22:59:14.039421 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:14.039422 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:14.039449 (+ 27us) server_negotiation.cc:407] Connection header received
0810 22:59:14.039521 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:14.039522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:14.039532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:14.039583 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:14.039892 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.040392 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.041142 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.041328 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.041379 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.049199 (+ 7820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:50097: BlockingRecv error: recv got EOF from 127.6.47.129:50097 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:14.183977 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:14.178201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:58097)
0810 22:59:14.178328 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:14.178339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:14.178403 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:14.178620 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:14.178621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:14.178934 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:14.179044 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.179048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.179696 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.179697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.180509 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.180512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.180998 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.180999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.181081 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.181629 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:14.181634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:14.183884 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
W20260810 22:59:14.184273 23059 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:14.178245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:58097 (local address 127.6.47.190:41851)
0810 22:59:14.178328 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:14.178330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:14.178386 (+ 56us) server_negotiation.cc:407] Connection header received
0810 22:59:14.178507 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:14.178508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:14.178517 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:14.178589 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:14.179117 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.179594 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.180610 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.180918 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.180998 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.184214 (+ 3216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:58097: BlockingRecv error: recv got EOF from 127.6.47.130:58097 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:14 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:14.321365 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:14.316243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:41681)
0810 22:59:14.316383 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:14.316395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:14.316455 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:14.316632 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:14.316633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:14.316801 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:14.316851 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.316852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.317537 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.317538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.318050 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:14.318051 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.318411 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.318412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:14.318452 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.318931 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:14.318936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:14.321241 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260810 22:59:14.321411 23059 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:14.316404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:41681 (local address 127.6.47.190:41851)
0810 22:59:14.316462 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:14.316464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:14.316476 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:59:14.316536 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:14.316537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:14.316552 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:14.316614 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:14.316934 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.317458 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.318142 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:14.318335 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:14.318401 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:14.321340 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:41681: BlockingRecv error: recv got EOF from 127.6.47.131:41681 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:15.054276 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:15.049728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:38791)
0810 22:59:15.049889 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:15.049900 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:15.049953 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:15.050082 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:15.050083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:15.050261 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:15.050310 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.050312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.050934 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.050935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.051487 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.051489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.051842 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.051843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.051895 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.052400 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:15.052405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:15.054216 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
W20260810 22:59:15.054409 23064 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:15.049815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:38791 (local address 127.6.47.190:41851)
0810 22:59:15.049924 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:15.049926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:15.049953 (+ 27us) server_negotiation.cc:407] Connection header received
0810 22:59:15.050003 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:15.050004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:15.050014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:15.050062 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:15.050377 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.050871 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.051557 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.051741 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.051810 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.054319 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:38791: BlockingRecv error: recv got EOF from 127.6.47.129:38791 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:15.189159 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:15.184418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:46299)
0810 22:59:15.184595 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:15.184607 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:15.184697 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:15.184889 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:15.184890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:15.185080 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:15.185127 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.185129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.185835 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.185837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.186395 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.186397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.186761 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.186762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.186832 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.187319 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:15.187324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:15.189088 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
W20260810 22:59:15.189265 23064 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:15.184488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:46299 (local address 127.6.47.190:41851)
0810 22:59:15.184583 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:15.184586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:15.184686 (+ 100us) server_negotiation.cc:407] Connection header received
0810 22:59:15.184783 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:15.184784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:15.184795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:15.184882 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:15.185242 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.185752 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.186486 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.186690 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.186763 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.189187 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:46299: BlockingRecv error: recv got EOF from 127.6.47.130:46299 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:15 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:15.326753 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:15.321850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:32945)
0810 22:59:15.321996 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:15.322006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:15.322065 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:15.322207 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:15.322208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:15.322377 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:15.322425 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.322427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.323070 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.323072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.323641 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:15.323643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.323986 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.323987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:15.324036 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.324480 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:15.324484 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:15.326650 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
W20260810 22:59:15.326784 23064 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:15.321928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:32945 (local address 127.6.47.190:41851)
0810 22:59:15.321982 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:15.321984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:15.322056 (+ 72us) server_negotiation.cc:407] Connection header received
0810 22:59:15.322113 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:15.322114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:15.322125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:15.322197 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:15.322486 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.322973 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.323732 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:15.323914 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:15.323968 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:15.326723 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:32945: BlockingRecv error: recv got EOF from 127.6.47.131:32945 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:16.059221 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:16.054815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:55635)
0810 22:59:16.054976 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:16.054988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:16.055064 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:16.055193 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:16.055194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:16.055362 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:16.055411 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.055412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.056036 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.056037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.056584 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.056585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.056918 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.056919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.056958 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.057473 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:16.057478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:16.059135 (+ 1657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
W20260810 22:59:16.059338 23075 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:16.054898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55635 (local address 127.6.47.190:41851)
0810 22:59:16.055007 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:16.055009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:16.055063 (+ 54us) server_negotiation.cc:407] Connection header received
0810 22:59:16.055110 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:16.055111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:16.055121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:16.055177 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:16.055472 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.055971 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.056656 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.056847 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.056906 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.059227 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:55635: BlockingRecv error: recv got EOF from 127.6.47.129:55635 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":51,"threads_started":1}
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:16.193847 23077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:16.189623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:39837)
0810 22:59:16.189760 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:16.189776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:16.189832 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:16.189941 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:16.189942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:16.190114 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:16.190162 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.190163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.190782 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.190783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.191288 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.191290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.191588 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.191589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.191627 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.192045 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:16.192049 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:16.193778 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
W20260810 22:59:16.193971 23075 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:16.189699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:39837 (local address 127.6.47.190:41851)
0810 22:59:16.189793 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:16.189795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:16.189816 (+ 21us) server_negotiation.cc:407] Connection header received
0810 22:59:16.189887 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:16.189888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:16.189898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:16.189938 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:16.190232 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.190720 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.191354 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.191528 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.191576 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.193872 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:39837: BlockingRecv error: recv got EOF from 127.6.47.130:39837 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:16 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:16.332086 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:16.327225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:56139)
0810 22:59:16.327458 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:16.327473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:16.327530 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:16.327679 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:16.327680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:16.327844 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:16.327892 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.327893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.328557 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.328558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.329379 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:16.329382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.329693 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.329694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:16.329734 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.330177 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:16.330182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:16.332016 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":129,"threads_started":1}
W20260810 22:59:16.332191 23075 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:16.327320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:56139 (local address 127.6.47.190:41851)
0810 22:59:16.327416 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:16.327424 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:16.327529 (+ 105us) server_negotiation.cc:407] Connection header received
0810 22:59:16.327598 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:16.327599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:16.327612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:16.327655 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:16.327968 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.328448 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.329446 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:16.329629 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:16.329691 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:16.332119 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:56139: BlockingRecv error: recv got EOF from 127.6.47.131:56139 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:17.068212 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:17.059796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:55935)
0810 22:59:17.059997 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:17.060008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:17.060079 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:17.060220 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:17.060220 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:17.060396 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:17.060446 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.060448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.061164 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.061176 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.061708 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.061709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.062060 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.062061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.062133 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.066362 (+ 4229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:17.066367 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:17.068132 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":120,"threads_started":1}
W20260810 22:59:17.068307 23081 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:17.059900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55935 (local address 127.6.47.190:41851)
0810 22:59:17.060026 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:17.060028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:17.060062 (+ 34us) server_negotiation.cc:407] Connection header received
0810 22:59:17.060138 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:17.060139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:17.060150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:17.060204 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:17.060547 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.061090 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.061786 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.061967 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.062053 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.068232 (+ 6179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:55935: BlockingRecv error: recv got EOF from 127.6.47.129:55935 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:17.198776 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:17.194367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:57351)
0810 22:59:17.194516 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:17.194528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:17.194593 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:17.194756 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:17.194757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:17.194926 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:17.194975 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.194976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.195600 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.195601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.196123 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.196125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.196444 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.196445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.196485 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.197024 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:17.197030 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:17.198718 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260810 22:59:17.198880 23081 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:17.194472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:57351 (local address 127.6.47.190:41851)
0810 22:59:17.194537 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:17.194539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:17.194583 (+ 44us) server_negotiation.cc:407] Connection header received
0810 22:59:17.194674 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:17.194675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:17.194684 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:17.194738 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:17.195037 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.195535 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.196200 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.196380 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.196429 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.198808 (+ 2379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:57351: BlockingRecv error: recv got EOF from 127.6.47.130:57351 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:17 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:17.337196 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:17.332703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:57587)
0810 22:59:17.332869 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:17.332882 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:17.332950 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:17.333057 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:17.333058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:17.333229 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:17.333277 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.333279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.333911 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.333911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.334434 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:17.334435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.334755 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.334756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:17.334795 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.335302 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:17.335306 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:17.337103 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260810 22:59:17.337299 23081 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:17.332764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:57587 (local address 127.6.47.190:41851)
0810 22:59:17.332812 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:17.332815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:17.332956 (+ 141us) server_negotiation.cc:407] Connection header received
0810 22:59:17.332994 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:17.332995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:17.333007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:17.333053 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:17.333338 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.333840 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.334506 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:17.334691 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:17.334743 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:17.337204 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:57587: BlockingRecv error: recv got EOF from 127.6.47.131:57587 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:18.073735 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:18.068857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:38011)
0810 22:59:18.069035 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:18.069047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:18.069102 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:18.069236 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:18.069236 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:18.069420 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:18.069469 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.069471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.070101 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.070102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.070651 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.070653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.070972 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.070973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.071013 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.071539 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:18.071544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:18.073675 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
W20260810 22:59:18.073879 23088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:18.068973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:38011 (local address 127.6.47.190:41851)
0810 22:59:18.069087 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:18.069089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:18.069098 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:18.069152 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:18.069152 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:18.069162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:18.069224 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:18.069533 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.070032 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.070721 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.070908 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.070958 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.073784 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:38011: BlockingRecv error: recv got EOF from 127.6.47.129:38011 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:18.203994 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:18.199297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:33225)
0810 22:59:18.199451 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:18.199468 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:18.199539 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:18.199684 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:18.199685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:18.199862 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:18.199907 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.199909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.200596 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.200597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.201095 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.201097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.201460 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.201461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.201517 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.202056 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:18.202061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:18.203897 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
W20260810 22:59:18.204043 23088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:18.199478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:33225 (local address 127.6.47.190:41851)
0810 22:59:18.199554 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:18.199556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:18.199565 (+ 9us) server_negotiation.cc:407] Connection header received
0810 22:59:18.199597 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:18.199600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:18.199616 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:18.199679 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:18.199987 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.200453 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.201207 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.201388 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.201461 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.203977 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:33225: BlockingRecv error: recv got EOF from 127.6.47.130:33225 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:18 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.131@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:18.342494 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:18.337697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.131:49633)
0810 22:59:18.337889 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:18.337901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:18.337979 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:18.338144 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:18.338145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:18.338352 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:18.338399 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.338402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.339101 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.339103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.339687 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:18.339689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.340089 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.340090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:18.340146 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.340649 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:18.340653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:18.342432 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":113,"threads_started":1}
W20260810 22:59:18.342670 23088 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:18.337796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:49633 (local address 127.6.47.190:41851)
0810 22:59:18.337872 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:18.337874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:18.337971 (+ 97us) server_negotiation.cc:407] Connection header received
0810 22:59:18.338047 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:18.338048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:18.338062 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:18.338136 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:18.338483 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.339016 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.339777 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:18.339982 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:18.340089 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:18.342569 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.131:49633: BlockingRecv error: recv got EOF from 127.6.47.131:49633 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 22:59:19 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:19 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.129@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:19.079316 23093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.074312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.129:45845)
0810 22:59:19.074577 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:19.074593 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:19.074687 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:19.074846 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:19.074847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:19.075022 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:19.075078 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.075080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.075767 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.075768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.076308 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.076310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.076808 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.076809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.076858 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.077295 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:19.077300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:19.079255 (+ 1955us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":107,"threads_started":1}
W20260810 22:59:19.079463 23095 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:19.074421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45845 (local address 127.6.47.190:41851)
0810 22:59:19.074554 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.074557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.074672 (+ 115us) server_negotiation.cc:407] Connection header received
0810 22:59:19.074746 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.074747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.074757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.074815 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.075148 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.075659 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.076399 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.076724 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.076794 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.079367 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.129:45845: BlockingRecv error: recv got EOF from 127.6.47.129:45845 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Aug 10 22:59:19 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 22:59:19 dist-test-slave-rwrg krb5kdc[21318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.47.130@KRBTEST.COM for oryx/127.6.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 22:59:19.209342 23097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.204594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41851 (local address 127.6.47.130:45909)
0810 22:59:19.204765 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:19.204775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:19.204839 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:19.204996 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:19.204997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:19.205197 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:19.205247 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.205249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.205900 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.205901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.206508 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.206510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.206826 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.206827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.206869 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.207404 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:19.207409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:19.209255 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.47.190:41851: Server oryx/127.6.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":69,"threads_started":1}
W20260810 22:59:19.209453 23095 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:19.204676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:45909 (local address 127.6.47.190:41851)
0810 22:59:19.204726 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.204728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.204849 (+ 121us) server_negotiation.cc:407] Connection header received
0810 22:59:19.204890 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.204891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.204906 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.204983 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.205316 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.205812 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.206581 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.206759 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.206807 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.209357 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.47.130:45909: BlockingRecv error: recv got EOF from 127.6.47.130:45909 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 22:59:19.265532 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 22283
I20260810 22:59:19.275799 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 22562
I20260810 22:59:19.286427 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 22795
I20260810 22:59:19.296904 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 22154
2026-08-10T22:59:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17835 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T22:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:19Z Disabled control of system clock
I20260810 22:59:19.332482 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34791
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:34791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 22:59:19.415895 23104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:19.416092 23104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:19.416110 23104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:19.417291 23104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:19.417361 23104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:19.417373 23104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:19.417387 23104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:19.417398 23104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:19.418710 23104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:34791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:19.418939 23104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:19.419198 23104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:19.423985 23119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.424146 23104 server_base.cc:1034] running on GCE node
W20260810 22:59:19.424019 23113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.424052 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.424628 23104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:19.424888 23104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:19.426061 23104 hybrid_clock.cc:648] HybridClock initialized: now 1786402759426054 us; error 39 us; skew 500 ppm
I20260810 22:59:19.427353 23104 webserver.cc:460] Webserver started at http://127.6.47.190:35293/ using document root <none> and password file <none>
I20260810 22:59:19.427627 23104 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:19.427703 23104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:19.427855 23104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:19.428970 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "b371682d08924ee7ae8a2db8aaae58a7"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "e9ce1a6c17dd50ee3d5def493eb6aafb"
server_key_iv: "fe9ff2d06dd4921cc703f0a758f28622"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.429375 23104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "b371682d08924ee7ae8a2db8aaae58a7"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "e9ce1a6c17dd50ee3d5def493eb6aafb"
server_key_iv: "fe9ff2d06dd4921cc703f0a758f28622"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.430666 23104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:19.431378 23126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.431540 23104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:19.431641 23104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "b371682d08924ee7ae8a2db8aaae58a7"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "e9ce1a6c17dd50ee3d5def493eb6aafb"
server_key_iv: "fe9ff2d06dd4921cc703f0a758f28622"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.431711 23104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:19.453037 23104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:19.453393 23104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:19.453558 23104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:19.458094 23104 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:34791
I20260810 22:59:19.458132 23226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:34791 every 8 connection(s)
I20260810 22:59:19.458515 23104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:59:19.459218 23228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:19.462029 23228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7: Bootstrap starting.
I20260810 22:59:19.462764 23228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:19.463073 23228 log.cc:824] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:19.463804 23228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7: No bootstrap required, opened a new log
I20260810 22:59:19.465586 23228 raft_consensus.cc:348] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } }
I20260810 22:59:19.465731 23228 raft_consensus.cc:374] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:19.465754 23228 raft_consensus.cc:729] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b371682d08924ee7ae8a2db8aaae58a7, State: Initialized, Role: FOLLOWER
I20260810 22:59:19.465934 23228 consensus_queue.cc:260] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } }
I20260810 22:59:19.465996 23228 raft_consensus.cc:388] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:59:19.466018 23228 raft_consensus.cc:482] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:59:19.466066 23228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:19.466631 23228 raft_consensus.cc:504] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } }
I20260810 22:59:19.466754 23228 leader_election.cc:302] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b371682d08924ee7ae8a2db8aaae58a7; no voters:
I20260810 22:59:19.466952 23228 leader_election.cc:288] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:59:19.467029 23233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:59:19.467227 23233 raft_consensus.cc:686] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [term 1 LEADER]: Becoming Leader. State: Replica: b371682d08924ee7ae8a2db8aaae58a7, State: Running, Role: LEADER
I20260810 22:59:19.467250 23228 sys_catalog.cc:564] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:59:19.467396 23233 consensus_queue.cc:237] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } }
I20260810 22:59:19.467435 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 23104
I20260810 22:59:19.467514 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:59:19.467728 6334 external_mini_cluster.cc:1428] Setting key c3e430463df77ac41777c563149c80d1
I20260810 22:59:19.467787 23237 sys_catalog.cc:455] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b371682d08924ee7ae8a2db8aaae58a7. Latest consensus state: current_term: 1 leader_uuid: "b371682d08924ee7ae8a2db8aaae58a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } } }
I20260810 22:59:19.467866 23237 sys_catalog.cc:458] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:19.467952 23235 sys_catalog.cc:455] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b371682d08924ee7ae8a2db8aaae58a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b371682d08924ee7ae8a2db8aaae58a7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34791 } } }
I20260810 22:59:19.468014 23235 sys_catalog.cc:458] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:19.468807 23245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:59:19.469592 23245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:59:19.471201 23245 catalog_manager.cc:1347] Generated new cluster ID: 8fa86a02b47c4fb0b46b3e618c4bdc1d
I20260810 22:59:19.471254 23245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:59:19.473881 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.468191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.6.47.190:34791)
0810 22:59:19.468657 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.468670 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.468710 (+ 40us) server_negotiation.cc:407] Connection header received
0810 22:59:19.468847 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.468855 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.468929 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.469096 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.469364 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.470581 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.471645 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.472234 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.472350 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.473177 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:19.473182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:19.473182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:19.473387 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:19.473426 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:19.473440 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:19.473502 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:19.473602 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 22:59:19.473677 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260810 22:59:19.486851 23245 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:59:19.487635 23245 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:59:19.491595 23245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b371682d08924ee7ae8a2db8aaae58a7: Generated new TSK 0
I20260810 22:59:19.491796 23245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:59:19.494410 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--builtin_ntp_servers=127.6.47.148:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 22:59:19.577720 23266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:19.577983 23266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:19.578019 23266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:19.579239 23266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:19.579300 23266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:19.579344 23266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:59:19.580761 23266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:19.581034 23266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:19.581403 23266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:19.586326 23272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.586338 23282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.586577 23266 server_base.cc:1034] running on GCE node
W20260810 22:59:19.586620 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.586831 23266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:19.587091 23266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:19.588307 23266 hybrid_clock.cc:648] HybridClock initialized: now 1786402759588314 us; error 68 us; skew 500 ppm
I20260810 22:59:19.589917 23266 webserver.cc:460] Webserver started at http://127.6.47.129:45111/ using document root <none> and password file <none>
I20260810 22:59:19.590178 23266 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:19.590248 23266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:19.590392 23266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:19.591475 23266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "c2130621528148db9cab100785986877"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "48c868c28861d8024440d0796948a27a"
server_key_iv: "8cb4b0743fa927b6cce16eadcb2bf43e"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.591881 23266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "c2130621528148db9cab100785986877"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "48c868c28861d8024440d0796948a27a"
server_key_iv: "8cb4b0743fa927b6cce16eadcb2bf43e"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.593379 23266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260810 22:59:19.594205 23293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.594383 23266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:19.594496 23266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "c2130621528148db9cab100785986877"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "48c868c28861d8024440d0796948a27a"
server_key_iv: "8cb4b0743fa927b6cce16eadcb2bf43e"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.594571 23266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:19.614622 23266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:19.614946 23266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:19.615115 23266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:19.615345 23266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:19.615684 23266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:19.615738 23266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.615780 23266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:19.615819 23266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.621945 23266 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:41727
I20260810 22:59:19.621991 23495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:41727 every 8 connection(s)
I20260810 22:59:19.622314 23266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:59:19.627840 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.623056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:39541 (local address 127.6.47.190:34791)
0810 22:59:19.623119 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.623121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.623387 (+ 266us) server_negotiation.cc:407] Connection header received
0810 22:59:19.624020 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.624021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.624038 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.624088 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.624872 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.625392 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.626360 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.626578 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.626656 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.627503 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:19.627504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:19.627504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:19.627510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:19.627524 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:19.627528 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:19.627600 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:19.627723 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 22:59:19.627764 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 22:59:19.628057 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.623051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:34791 (local address 127.6.47.129:39541)
0810 22:59:19.623262 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:19.623296 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:19.623901 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:19.624162 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:19.624166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:19.624183 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:19.624751 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.624757 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.625492 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.625493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.626257 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.626259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.626669 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.626670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.626802 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.627347 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:19.627348 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:19.627375 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:19.627397 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:19.627602 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:19.627608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:19.627612 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:59:19.627713 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 22:59:19.627825 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260810 22:59:19.628618 23498 heartbeater.cc:347] Connected to a master server at 127.6.47.190:34791
I20260810 22:59:19.628701 23498 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:19.628839 23498 heartbeater.cc:511] Master 127.6.47.190:34791 requested a full tablet report, sending...
I20260810 22:59:19.629364 23159 ts_manager.cc:194] Registered new tserver with Master: c2130621528148db9cab100785986877 (127.6.47.129:41727)
I20260810 22:59:19.630003 23159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.129:39541
I20260810 22:59:19.630429 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 23266
I20260810 22:59:19.630518 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:59:19.630720 6334 external_mini_cluster.cc:1428] Setting key 62e242e8a24bf2286e6afa5343628850
I20260810 22:59:19.631628 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--builtin_ntp_servers=127.6.47.148:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 22:59:19.716842 23502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:19.717036 23502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:19.717057 23502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:19.718201 23502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:19.718241 23502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:19.718266 23502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:59:19.719595 23502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:19.719820 23502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:19.720122 23502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:19.725158 23519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.725158 23513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.725275 23514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.725430 23502 server_base.cc:1034] running on GCE node
I20260810 22:59:19.725652 23502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:19.725907 23502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:19.727068 23502 hybrid_clock.cc:648] HybridClock initialized: now 1786402759727065 us; error 41 us; skew 500 ppm
I20260810 22:59:19.728314 23502 webserver.cc:460] Webserver started at http://127.6.47.130:37973/ using document root <none> and password file <none>
I20260810 22:59:19.728657 23502 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:19.728735 23502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:19.728875 23502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:19.729960 23502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "b3be9343a72a4c40b4e1f60cd93ae5aa"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "cfbba014f390f2d9e8bd1fdb4b4bd609"
server_key_iv: "04d8c8c40357176bac623dad4c12091f"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.730329 23502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "b3be9343a72a4c40b4e1f60cd93ae5aa"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "cfbba014f390f2d9e8bd1fdb4b4bd609"
server_key_iv: "04d8c8c40357176bac623dad4c12091f"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.731719 23502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:59:19.732465 23528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.732684 23502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 22:59:19.732786 23502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "b3be9343a72a4c40b4e1f60cd93ae5aa"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "cfbba014f390f2d9e8bd1fdb4b4bd609"
server_key_iv: "04d8c8c40357176bac623dad4c12091f"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.732872 23502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:19.756557 23502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:19.756927 23502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:19.757082 23502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:19.757328 23502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:19.757688 23502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:19.757743 23502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.757786 23502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:19.757813 23502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.766222 23502 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:33559
I20260810 22:59:19.766278 23728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:33559 every 8 connection(s)
I20260810 22:59:19.766613 23502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:59:19.767611 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 23502
I20260810 22:59:19.767712 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:59:19.767935 6334 external_mini_cluster.cc:1428] Setting key e5918a3ed9bad8f3c29735f16161fc23
I20260810 22:59:19.769047 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--builtin_ntp_servers=127.6.47.148:41883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 22:59:19.772850 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.767373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:49433 (local address 127.6.47.190:34791)
0810 22:59:19.767474 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.767477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.767765 (+ 288us) server_negotiation.cc:407] Connection header received
0810 22:59:19.768463 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.768464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.768474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.768572 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.769532 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.769995 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.771362 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.771683 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.771757 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.772620 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:19.772621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:19.772622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:19.772626 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:19.772640 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:19.772644 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:19.772695 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:19.772771 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 22:59:19.772793 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 22:59:19.773140 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.767375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:34791 (local address 127.6.47.130:49433)
0810 22:59:19.767626 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:19.767677 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:19.768370 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:19.768673 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:19.768676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:19.768685 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:19.769437 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.769445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.770115 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.770116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.771269 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.771272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.771777 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.771778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.771887 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.772449 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:19.772449 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:19.772511 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:19.772540 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:19.772701 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:19.772705 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:19.772708 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:59:19.772774 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 22:59:19.772885 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260810 22:59:19.773684 23729 heartbeater.cc:347] Connected to a master server at 127.6.47.190:34791
I20260810 22:59:19.773806 23729 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:19.773993 23729 heartbeater.cc:511] Master 127.6.47.190:34791 requested a full tablet report, sending...
I20260810 22:59:19.774398 23159 ts_manager.cc:194] Registered new tserver with Master: b3be9343a72a4c40b4e1f60cd93ae5aa (127.6.47.130:33559)
I20260810 22:59:19.774782 23159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.130:49433
W20260810 22:59:19.855513 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:19.855754 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:19.855788 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:19.856977 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:19.857038 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:19.857079 23734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:59:19.858457 23734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:34791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:19.858731 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:19.859032 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:19.864135 23745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.864090 23752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:19.864071 23749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:19.864392 23734 server_base.cc:1034] running on GCE node
I20260810 22:59:19.864677 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:19.864964 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:19.866206 23734 hybrid_clock.cc:648] HybridClock initialized: now 1786402759866180 us; error 44 us; skew 500 ppm
I20260810 22:59:19.867978 23734 webserver.cc:460] Webserver started at http://127.6.47.131:42171/ using document root <none> and password file <none>
I20260810 22:59:19.868256 23734 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:19.868319 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:19.868517 23734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:19.869871 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "e0b379b4236a4db4b16dba34577e2d3f"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "58f675164bba4fdd5b0e2eae1d81121e"
server_key_iv: "5e4004e73622ebfab1b0899147a981bf"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.870271 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "e0b379b4236a4db4b16dba34577e2d3f"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "58f675164bba4fdd5b0e2eae1d81121e"
server_key_iv: "5e4004e73622ebfab1b0899147a981bf"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.871940 23734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:19.872948 23760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.873142 23734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:19.873242 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "e0b379b4236a4db4b16dba34577e2d3f"
format_stamp: "Formatted at 2026-08-10 22:59:19 on dist-test-slave-rwrg"
server_key: "58f675164bba4fdd5b0e2eae1d81121e"
server_key_iv: "5e4004e73622ebfab1b0899147a981bf"
server_key_version: "encryptionkey@0"
I20260810 22:59:19.873337 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:19.900630 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:19.901008 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:19.901166 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:19.901448 23734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:19.901820 23734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:19.901878 23734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.901925 23734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:19.901957 23734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:19.908344 23734 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:46367
I20260810 22:59:19.908387 23967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:46367 every 8 connection(s)
I20260810 22:59:19.908816 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:59:19.914340 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.909618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:57505 (local address 127.6.47.190:34791)
0810 22:59:19.909665 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.909679 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.909923 (+ 244us) server_negotiation.cc:407] Connection header received
0810 22:59:19.910523 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.910524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.910538 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.910589 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.911309 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.911824 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.912947 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.913142 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.913223 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.914023 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:19.914024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:19.914025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:19.914030 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:19.914045 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:19.914049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:19.914118 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:19.914226 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 22:59:19.914258 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 22:59:19.914588 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:19.909593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:34791 (local address 127.6.47.131:57505)
0810 22:59:19.909797 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:19.909834 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:19.910401 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:19.910661 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:19.910666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:19.910676 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:19.911201 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.911205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.911925 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.911926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.912840 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:19.912843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:19.913233 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:19.913234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:19.913370 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:19.913881 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:19.913881 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:19.913911 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:19.913933 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:19.914119 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:19.914126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:19.914130 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:59:19.914228 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 22:59:19.914331 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260810 22:59:19.915105 23968 heartbeater.cc:347] Connected to a master server at 127.6.47.190:34791
I20260810 22:59:19.915205 23968 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:19.915354 23968 heartbeater.cc:511] Master 127.6.47.190:34791 requested a full tablet report, sending...
I20260810 22:59:19.915741 23159 ts_manager.cc:194] Registered new tserver with Master: e0b379b4236a4db4b16dba34577e2d3f (127.6.47.131:46367)
I20260810 22:59:19.915779 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 23734
I20260810 22:59:19.915874 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:59:19.916072 6334 external_mini_cluster.cc:1428] Setting key 72dc5f3c619065f77124048437ab3834
I20260810 22:59:19.916190 23159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.131:57505
I20260810 22:59:19.917062 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:59:19.918879 23249 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:19.918359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.6.47.190:34791)
0810 22:59:19.918441 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:19.918443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:19.918538 (+ 95us) server_negotiation.cc:407] Connection header received
0810 22:59:19.918611 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:19.918612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:19.918626 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:19.918670 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:19.918815 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34392: BlockingRecv error: recv got EOF from 127.0.0.1:34392 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 22:59:19.918934 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 23266
I20260810 22:59:19.927075 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 23502
I20260810 22:59:19.939703 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 23734
I20260810 22:59:19.950790 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 23104
2026-08-10T22:59:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (652 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T22:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:19Z Disabled control of system clock
I20260810 22:59:19.984886 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35477
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:35477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 22:59:20.069124 23983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:20.069361 23983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:20.069386 23983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:20.070473 23983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:20.070521 23983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:20.070537 23983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:20.070551 23983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:20.070565 23983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:20.071888 23983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:35477
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:20.072090 23983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:20.072348 23983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:20.077387 23992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.077399 23991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:20.077488 23983 server_base.cc:1034] running on GCE node
W20260810 22:59:20.077436 23995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:20.077925 23983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:20.078292 23983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:20.079454 23983 hybrid_clock.cc:648] HybridClock initialized: now 1786402760079429 us; error 42 us; skew 500 ppm
I20260810 22:59:20.080892 23983 webserver.cc:460] Webserver started at http://127.6.47.190:39427/ using document root <none> and password file <none>
I20260810 22:59:20.081167 23983 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:20.081218 23983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:20.081395 23983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:20.082532 23983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "f1c03b16642f4f8fa6f95642fabb16fd"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9a971d91c635cff8c3b1e1e0ce803e94"
server_key_iv: "be407aaa572ca6ea9938bd66abcd9f23"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.082909 23983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "f1c03b16642f4f8fa6f95642fabb16fd"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9a971d91c635cff8c3b1e1e0ce803e94"
server_key_iv: "be407aaa572ca6ea9938bd66abcd9f23"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.084370 23983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:20.085210 24004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.085395 23983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:59:20.085494 23983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "f1c03b16642f4f8fa6f95642fabb16fd"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9a971d91c635cff8c3b1e1e0ce803e94"
server_key_iv: "be407aaa572ca6ea9938bd66abcd9f23"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.085572 23983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:20.089051 23983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:20.089362 23983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:20.089501 23983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:20.093585 23983 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:35477
I20260810 22:59:20.093629 24084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:35477 every 8 connection(s)
I20260810 22:59:20.093978 23983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:59:20.094687 24085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:20.097584 24085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd: Bootstrap starting.
I20260810 22:59:20.098172 24085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:20.098486 24085 log.cc:824] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:20.098987 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 23983
I20260810 22:59:20.099105 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:59:20.099282 6334 external_mini_cluster.cc:1428] Setting key b0bd37bbec1fe5d2e99bcbcae4aa14be
I20260810 22:59:20.099278 24085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd: No bootstrap required, opened a new log
I20260810 22:59:20.101953 24085 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } }
I20260810 22:59:20.102151 24085 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:20.102188 24085 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1c03b16642f4f8fa6f95642fabb16fd, State: Initialized, Role: FOLLOWER
I20260810 22:59:20.102238 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.099866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41376 (local address 127.6.47.190:35477)
0810 22:59:20.100109 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:20.100118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:20.100150 (+ 32us) server_negotiation.cc:407] Connection header received
0810 22:59:20.100794 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:20.100804 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:20.100854 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:20.101009 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:20.101767 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:20.101771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:20.101772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:20.101777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:20.101828 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:20.101841 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:20.101936 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:20.101997 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 22:59:20.102055 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260810 22:59:20.102311 24085 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } }
I20260810 22:59:20.102408 24085 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:59:20.102447 24085 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:59:20.102501 24085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:20.103466 24085 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } }
I20260810 22:59:20.103574 24085 leader_election.cc:302] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1c03b16642f4f8fa6f95642fabb16fd; no voters:
I20260810 22:59:20.103708 24085 leader_election.cc:288] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:59:20.103780 24093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:59:20.103964 24093 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [term 1 LEADER]: Becoming Leader. State: Replica: f1c03b16642f4f8fa6f95642fabb16fd, State: Running, Role: LEADER
I20260810 22:59:20.103989 24085 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:59:20.104059 24093 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } }
I20260810 22:59:20.104415 24097 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1c03b16642f4f8fa6f95642fabb16fd. Latest consensus state: current_term: 1 leader_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } } }
I20260810 22:59:20.104655 24097 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:20.104411 24096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1c03b16642f4f8fa6f95642fabb16fd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35477 } } }
I20260810 22:59:20.104723 24096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:20.104892 24106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:59:20.105435 24106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:59:20.106889 24106 catalog_manager.cc:1347] Generated new cluster ID: ef9c462e1bc24a5e9e40e4087ed8bf1c
I20260810 22:59:20.106947 24106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 22:59:20.115614 24106 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:59:20.115789 24106 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:59:20.127736 24106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1c03b16642f4f8fa6f95642fabb16fd: Generated new TSK 0
I20260810 22:59:20.127947 24106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 22:59:20.138168 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--builtin_ntp_servers=127.6.47.148:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 22:59:20.246093 24124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:20.246297 24124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:20.246318 24124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:20.247442 24124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:20.247479 24124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:20.247502 24124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:59:20.248793 24124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:20.249032 24124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:20.249358 24124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:20.255177 24150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.255201 24139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.255223 24148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:20.255460 24124 server_base.cc:1034] running on GCE node
I20260810 22:59:20.255679 24124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:20.255931 24124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:20.257081 24124 hybrid_clock.cc:648] HybridClock initialized: now 1786402760257065 us; error 34 us; skew 500 ppm
I20260810 22:59:20.258407 24124 webserver.cc:460] Webserver started at http://127.6.47.129:35895/ using document root <none> and password file <none>
I20260810 22:59:20.258635 24124 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:20.258682 24124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:20.258821 24124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:20.259855 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "68d98adc728e4409a347936645648b6c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9b92cbfde6de34d3375fab38af48b576"
server_key_iv: "f55a03554a31fe018414c60cbdf5f4cf"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.260218 24124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "68d98adc728e4409a347936645648b6c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9b92cbfde6de34d3375fab38af48b576"
server_key_iv: "f55a03554a31fe018414c60cbdf5f4cf"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.261642 24124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:20.262362 24165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.262543 24124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:20.262640 24124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "68d98adc728e4409a347936645648b6c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "9b92cbfde6de34d3375fab38af48b576"
server_key_iv: "f55a03554a31fe018414c60cbdf5f4cf"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.262722 24124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:20.266085 24124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:20.266347 24124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:20.266494 24124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:20.266723 24124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:20.267020 24124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:20.267069 24124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.267107 24124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:20.267133 24124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.273245 24124 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40743
I20260810 22:59:20.273283 24356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40743 every 8 connection(s)
I20260810 22:59:20.273598 24124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 22:59:20.274705 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 24124
I20260810 22:59:20.274801 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:59:20.274981 6334 external_mini_cluster.cc:1428] Setting key b1b8e1d7ccf41ef91d75811285629f5c
I20260810 22:59:20.275987 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--builtin_ntp_servers=127.6.47.148:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 22:59:20.276728 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.274343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:59941 (local address 127.6.47.190:35477)
0810 22:59:20.274389 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:20.274391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:20.274596 (+ 205us) server_negotiation.cc:407] Connection header received
0810 22:59:20.275126 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:20.275128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:20.275135 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:20.275200 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:20.276436 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:20.276438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:20.276438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:20.276443 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:20.276459 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:20.276464 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:20.276544 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:20.276613 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 22:59:20.276629 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 22:59:20.276851 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.274325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35477 (local address 127.6.47.129:59941)
0810 22:59:20.274491 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:20.274522 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:20.275012 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:20.275276 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:20.275280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:20.275289 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:20.276295 (+ 1006us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:20.276296 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:20.276322 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:20.276349 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:20.276544 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:20.276548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:20.276551 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:59:20.276612 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 22:59:20.276660 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260810 22:59:20.277531 24358 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35477
I20260810 22:59:20.277619 24358 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:20.277768 24358 heartbeater.cc:511] Master 127.6.47.190:35477 requested a full tablet report, sending...
I20260810 22:59:20.278324 24026 ts_manager.cc:194] Registered new tserver with Master: 68d98adc728e4409a347936645648b6c (127.6.47.129:40743)
W20260810 22:59:20.360610 24362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:20.360874 24362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:20.360896 24362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:20.362092 24362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:20.362133 24362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:20.362159 24362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:59:20.363451 24362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:20.363672 24362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:20.363945 24362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:20.369107 24371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.369128 24375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.369154 24372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:20.369451 24362 server_base.cc:1034] running on GCE node
I20260810 22:59:20.369668 24362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:20.369868 24362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:20.371011 24362 hybrid_clock.cc:648] HybridClock initialized: now 1786402760370993 us; error 33 us; skew 500 ppm
I20260810 22:59:20.372315 24362 webserver.cc:460] Webserver started at http://127.6.47.130:33589/ using document root <none> and password file <none>
I20260810 22:59:20.372602 24362 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:20.372653 24362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:20.372798 24362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:20.373893 24362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "49d3b9a5bb2842f3bd859ecb6414203c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "85900b6a9da81ded13b01333bd5ba7a2"
server_key_iv: "26aa61cb99456596315e5ff4fb9dd4e7"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.374291 24362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "49d3b9a5bb2842f3bd859ecb6414203c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "85900b6a9da81ded13b01333bd5ba7a2"
server_key_iv: "26aa61cb99456596315e5ff4fb9dd4e7"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.375631 24362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 22:59:20.376401 24391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.376585 24362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 22:59:20.376684 24362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "49d3b9a5bb2842f3bd859ecb6414203c"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "85900b6a9da81ded13b01333bd5ba7a2"
server_key_iv: "26aa61cb99456596315e5ff4fb9dd4e7"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.376762 24362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:20.380070 24362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:20.380400 24362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:20.380561 24362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:20.380781 24362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:20.381093 24362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:20.381127 24362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.381147 24362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:20.381160 24362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.387085 24362 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:46581
I20260810 22:59:20.387159 24598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:46581 every 8 connection(s)
I20260810 22:59:20.387485 24362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 22:59:20.390802 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.388257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:36367 (local address 127.6.47.190:35477)
0810 22:59:20.388308 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:20.388312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:20.388587 (+ 275us) server_negotiation.cc:407] Connection header received
0810 22:59:20.389117 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:20.389119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:20.389125 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:20.389153 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:20.390528 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:20.390529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:20.390530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:20.390534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:20.390548 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:20.390553 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:20.390601 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:20.390714 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 22:59:20.390730 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 22:59:20.390996 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.388252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35477 (local address 127.6.47.130:36367)
0810 22:59:20.388440 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:20.388473 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:20.388967 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:20.389365 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:20.389369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:20.389379 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:20.390350 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:20.390351 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:20.390380 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:20.390408 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:20.390598 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:20.390612 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:20.390615 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:59:20.390725 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 22:59:20.390790 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260810 22:59:20.391546 24599 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35477
I20260810 22:59:20.391666 24599 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:20.391822 24599 heartbeater.cc:511] Master 127.6.47.190:35477 requested a full tablet report, sending...
I20260810 22:59:20.392175 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 24362
I20260810 22:59:20.392190 24026 ts_manager.cc:194] Registered new tserver with Master: 49d3b9a5bb2842f3bd859ecb6414203c (127.6.47.130:46581)
I20260810 22:59:20.392283 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:59:20.392561 6334 external_mini_cluster.cc:1428] Setting key afba2140b78237c7399a391997718d88
I20260810 22:59:20.393438 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--builtin_ntp_servers=127.6.47.148:36719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 22:59:20.477324 24602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:20.477559 24602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:20.477597 24602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:20.478726 24602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:20.478793 24602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:20.478832 24602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:59:20.480314 24602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:20.480603 24602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:20.480947 24602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:20.486032 24609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.486029 24615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:20.486008 24610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:20.486297 24602 server_base.cc:1034] running on GCE node
I20260810 22:59:20.486572 24602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:20.486779 24602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:20.487926 24602 hybrid_clock.cc:648] HybridClock initialized: now 1786402760487913 us; error 29 us; skew 500 ppm
I20260810 22:59:20.489411 24602 webserver.cc:460] Webserver started at http://127.6.47.131:38599/ using document root <none> and password file <none>
I20260810 22:59:20.489655 24602 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:20.489711 24602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:20.489805 24602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:20.490886 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "4e632cc29323403ba07da2c09c80745d"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "bb865a9845a6e77908dfbe0b2004350a"
server_key_iv: "4b4847d42f11a111375c559d5f9f805b"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.491206 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "4e632cc29323403ba07da2c09c80745d"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "bb865a9845a6e77908dfbe0b2004350a"
server_key_iv: "4b4847d42f11a111375c559d5f9f805b"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.492583 24602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:20.493316 24624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.493501 24602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:20.493597 24602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "4e632cc29323403ba07da2c09c80745d"
format_stamp: "Formatted at 2026-08-10 22:59:20 on dist-test-slave-rwrg"
server_key: "bb865a9845a6e77908dfbe0b2004350a"
server_key_iv: "4b4847d42f11a111375c559d5f9f805b"
server_key_version: "encryptionkey@0"
I20260810 22:59:20.493685 24602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:20.497066 24602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:20.497330 24602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:20.497707 24602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:20.497942 24602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:20.498271 24602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:20.498328 24602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.498369 24602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:20.498396 24602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:20.504571 24602 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:35405
I20260810 22:59:20.504675 24819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:35405 every 8 connection(s)
I20260810 22:59:20.504998 24602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 22:59:20.507973 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.505864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:45471 (local address 127.6.47.190:35477)
0810 22:59:20.505937 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:20.505939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:20.506148 (+ 209us) server_negotiation.cc:407] Connection header received
0810 22:59:20.506616 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:20.506617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:20.506624 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:20.506703 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:20.507771 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:20.507772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:20.507772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 22:59:20.507777 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:20.507793 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 22:59:20.507797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:20.507857 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:20.507904 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 22:59:20.507919 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 22:59:20.508179 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:20.505853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35477 (local address 127.6.47.131:45471)
0810 22:59:20.506042 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:20.506076 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:20.506507 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:20.506757 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:20.506760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:20.506768 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:20.507624 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 22:59:20.507624 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:20.507653 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 22:59:20.507679 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:20.507857 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:20.507859 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:20.507862 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 22:59:20.507910 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 22:59:20.507954 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":103,"threads_started":1}
I20260810 22:59:20.508802 24821 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35477
I20260810 22:59:20.508927 24821 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:20.509074 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 24602
I20260810 22:59:20.509105 24821 heartbeater.cc:511] Master 127.6.47.190:35477 requested a full tablet report, sending...
I20260810 22:59:20.509159 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:59:20.509368 6334 external_mini_cluster.cc:1428] Setting key 91ac70b26f8ccd5322f594210a2e1f20
I20260810 22:59:20.509552 24026 ts_manager.cc:194] Registered new tserver with Master: 4e632cc29323403ba07da2c09c80745d (127.6.47.131:35405)
I20260810 22:59:20.510212 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 22:59:20.512009 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 22:59:20.511527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.6.47.190:35477)
0810 22:59:20.511615 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:20.511617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:20.511702 (+ 85us) server_negotiation.cc:407] Connection header received
0810 22:59:20.511747 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:20.511748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:20.511752 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:20.511808 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:20.511941 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41388: BlockingRecv error: recv got EOF from 127.0.0.1:41388 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 22:59:20.512048 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 24124
I20260810 22:59:20.525563 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 24362
I20260810 22:59:20.539625 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 24602
I20260810 22:59:20.550859 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 23983
2026-08-10T22:59:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (601 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:59:20 dist-test-slave-rwrg krb5kdc[24837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:59:20 dist-test-slave-rwrg krb5kdc[24837](info): set up 2 sockets
Aug 10 22:59:20 dist-test-slave-rwrg krb5kdc[24837](info): commencing operation
krb5kdc: starting...
W20260810 22:59:21.844444 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:59:21 dist-test-slave-rwrg krb5kdc[24837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:59:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 22:59:21.966747 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42387
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:22.048779 24861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:22.048983 24861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:22.049006 24861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:22.050081 24861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 22:59:22.050117 24861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:22.050132 24861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:22.050144 24861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 22:59:22.050156 24861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 22:59:22.051544 24861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:22.051787 24861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:22.051985 24861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:22.056851 24871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.056903 24872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.056864 24876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:22.056949 24861 server_base.cc:1034] running on GCE node
I20260810 22:59:22.057281 24861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:22.057614 24861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:22.058784 24861 hybrid_clock.cc:648] HybridClock initialized: now 1786402762058775 us; error 48 us; skew 500 ppm
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:22.061405 24861 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 22:59:22.061961 24861 webserver.cc:460] Webserver started at http://127.6.47.190:43925/ using document root <none> and password file <none>
I20260810 22:59:22.062217 24861 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:22.062284 24861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:22.062448 24861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:22.063529 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "2aa1571c74b94256945e28a39fa40093"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "9396f4bec8954632794114561f8adfa0"
server_key_iv: "1e63ceff8c773f200c7f1fa4a20445d1"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.063908 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "2aa1571c74b94256945e28a39fa40093"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "9396f4bec8954632794114561f8adfa0"
server_key_iv: "1e63ceff8c773f200c7f1fa4a20445d1"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.065373 24861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:22.066094 24889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.066280 24861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:22.066393 24861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2aa1571c74b94256945e28a39fa40093"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "9396f4bec8954632794114561f8adfa0"
server_key_iv: "1e63ceff8c773f200c7f1fa4a20445d1"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.066468 24861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:22.078392 24861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:22.080788 24861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:22.080945 24861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:22.085258 24861 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42387
I20260810 22:59:22.085290 24988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42387 every 8 connection(s)
I20260810 22:59:22.085702 24861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 22:59:22.086767 24989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:22.089133 24989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: Bootstrap starting.
I20260810 22:59:22.089706 24989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:22.089941 24989 log.cc:824] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:22.090601 24989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: No bootstrap required, opened a new log
I20260810 22:59:22.092163 24989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } }
I20260810 22:59:22.092298 24989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:22.092319 24989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aa1571c74b94256945e28a39fa40093, State: Initialized, Role: FOLLOWER
I20260810 22:59:22.092480 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 24861
I20260810 22:59:22.092486 24989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } }
I20260810 22:59:22.092602 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 22:59:22.092633 24989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 22:59:22.092674 24989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 22:59:22.092722 24989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:22.092798 6334 external_mini_cluster.cc:1428] Setting key b9bcde94e2bf6c18536b3e7c35a0f58a
I20260810 22:59:22.093731 24989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } }
I20260810 22:59:22.093873 24989 leader_election.cc:302] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aa1571c74b94256945e28a39fa40093; no voters:
I20260810 22:59:22.094051 24989 leader_election.cc:288] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 22:59:22.094074 24997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:59:22.094267 24997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [term 1 LEADER]: Becoming Leader. State: Replica: 2aa1571c74b94256945e28a39fa40093, State: Running, Role: LEADER
I20260810 22:59:22.094328 24989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 22:59:22.094404 24997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } }
I20260810 22:59:22.094802 25001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aa1571c74b94256945e28a39fa40093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } } }
I20260810 22:59:22.094862 25001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:22.094772 25002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aa1571c74b94256945e28a39fa40093. Latest consensus state: current_term: 1 leader_uuid: "2aa1571c74b94256945e28a39fa40093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aa1571c74b94256945e28a39fa40093" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42387 } } }
I20260810 22:59:22.094998 25002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093 [sys.catalog]: This master's current role is: LEADER
I20260810 22:59:22.095216 25010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 22:59:22.095965 25010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 22:59:22.097806 25010 catalog_manager.cc:1347] Generated new cluster ID: 6c8c6f28e60842e48ccc88a0eb0dd946
I20260810 22:59:22.097893 25010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:22.105155 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.093332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.6.47.190:42387)
0810 22:59:22.093514 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.093522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.093551 (+ 29us) server_negotiation.cc:407] Connection header received
0810 22:59:22.093645 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.093651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.093693 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.093800 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:22.094645 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.096205 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.097102 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.097681 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.097791 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.100866 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:22.100870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:22.100871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:22.100884 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:22.103008 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.103427 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.103430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.103431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.103548 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.103777 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.103778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.103779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.104174 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:22.104318 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.104792 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.104936 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260810 22:59:22.127769 25010 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 22:59:22.128319 25010 catalog_manager.cc:1504] Loading token signing keys...
I20260810 22:59:22.132146 25010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: Generated new TSK 0
I20260810 22:59:22.132350 25010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 22:59:22.161092 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:22.244798 25038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:22.245015 25038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:22.245040 25038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:22.246291 25038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:22.246337 25038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:22.246366 25038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 22:59:22.247911 25038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:22.248186 25038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:22.248435 25038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:22.253654 25048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.253654 25045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.253705 25046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:22.254109 25038 server_base.cc:1034] running on GCE node
I20260810 22:59:22.254369 25038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:22.254678 25038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:22.255842 25038 hybrid_clock.cc:648] HybridClock initialized: now 1786402762255818 us; error 41 us; skew 500 ppm
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:22.258625 25038 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 22:59:22.259127 25038 webserver.cc:460] Webserver started at http://127.6.47.129:40355/ using document root <none> and password file <none>
I20260810 22:59:22.259395 25038 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:22.259470 25038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:22.259634 25038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:22.261122 25038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "392f8c2991a842388bd0fa4430b94560"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ec11b0b73d7e4c0d4f218e46a5f1f6a2"
server_key_iv: "01c7322a0c9456e5f74785e84ba32610"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.261524 25038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "392f8c2991a842388bd0fa4430b94560"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ec11b0b73d7e4c0d4f218e46a5f1f6a2"
server_key_iv: "01c7322a0c9456e5f74785e84ba32610"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.262868 25038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 22:59:22.263590 25061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.263773 25038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:22.263886 25038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "392f8c2991a842388bd0fa4430b94560"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ec11b0b73d7e4c0d4f218e46a5f1f6a2"
server_key_iv: "01c7322a0c9456e5f74785e84ba32610"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.263965 25038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:22.279884 25038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:22.281723 25038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:22.281899 25038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:22.282159 25038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:22.282500 25038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:22.282562 25038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.282606 25038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:22.282656 25038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.289795 25038 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:33371
I20260810 22:59:22.289834 25274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:33371 every 8 connection(s)
I20260810 22:59:22.290184 25038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:22.297014 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 25038
I20260810 22:59:22.297135 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 22:59:22.297343 6334 external_mini_cluster.cc:1428] Setting key c63b9a9d17546627650ba46c8fdbdc88
I20260810 22:59:22.300804 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.290992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45069 (local address 127.6.47.190:42387)
0810 22:59:22.291051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.291053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.291271 (+ 218us) server_negotiation.cc:407] Connection header received
0810 22:59:22.291807 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.291808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.291824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.291891 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:22.292900 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.293422 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.294499 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.294707 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.294781 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.297487 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:22.297489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:22.297490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:22.297499 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:22.299116 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.299713 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.299714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.299715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.299793 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.300072 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.300073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.300074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.300221 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:22.300294 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.300569 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.300681 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 22:59:22.300846 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.290958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42387 (local address 127.6.47.129:45069)
0810 22:59:22.291153 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.291189 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.291701 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.291956 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.291960 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.292614 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:22.292801 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.292807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.293523 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.293524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.294418 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.294420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.294802 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.294804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.294963 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.295674 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:22.295682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:22.297313 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:22.299212 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.299221 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.299223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.299563 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.299884 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.299884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.299885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.300001 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.300289 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:22.300294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:22.300409 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.300535 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.300684 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260810 22:59:22.301409 25276 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42387
I20260810 22:59:22.301510 25276 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:22.301710 25276 heartbeater.cc:511] Master 127.6.47.190:42387 requested a full tablet report, sending...
I20260810 22:59:22.302217 24923 ts_manager.cc:194] Registered new tserver with Master: 392f8c2991a842388bd0fa4430b94560 (127.6.47.129:33371)
I20260810 22:59:22.302889 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:45069
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 22:59:22.325603 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:22.409030 25290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:22.409287 25290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:22.409327 25290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:22.410502 25290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:22.410569 25290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:22.410612 25290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 22:59:22.411968 25290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:22.412241 25290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:22.412458 25290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:22.417462 25295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.417454 25299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.417454 25303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:22.417735 25290 server_base.cc:1034] running on GCE node
I20260810 22:59:22.417949 25290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:22.418196 25290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:22.419361 25290 hybrid_clock.cc:648] HybridClock initialized: now 1786402762419334 us; error 43 us; skew 500 ppm
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:22.421851 25290 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 22:59:22.422331 25290 webserver.cc:460] Webserver started at http://127.6.47.130:32807/ using document root <none> and password file <none>
I20260810 22:59:22.422545 25290 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:22.422612 25290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:22.422761 25290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:22.423787 25290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "95113b8315784e3fae82f2ec7b80d62b"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ab8375036563a26c401c36683128335b"
server_key_iv: "42bd252282cc1413a3f014a042a77a98"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.424162 25290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "95113b8315784e3fae82f2ec7b80d62b"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ab8375036563a26c401c36683128335b"
server_key_iv: "42bd252282cc1413a3f014a042a77a98"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.425561 25290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:22.426290 25313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.426469 25290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 22:59:22.426575 25290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "95113b8315784e3fae82f2ec7b80d62b"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "ab8375036563a26c401c36683128335b"
server_key_iv: "42bd252282cc1413a3f014a042a77a98"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.426648 25290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:22.453567 25290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:22.455497 25290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:22.455669 25290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:22.455927 25290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:22.456249 25290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:22.456306 25290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.456347 25290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:22.456377 25290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.462932 25290 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:43931
I20260810 22:59:22.462984 25501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:43931 every 8 connection(s)
I20260810 22:59:22.463307 25290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:22.471472 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 25290
I20260810 22:59:22.471573 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 22:59:22.471753 6334 external_mini_cluster.cc:1428] Setting key 81a95f294f4988466a361c421b021971
I20260810 22:59:22.472569 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.464070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:46547 (local address 127.6.47.190:42387)
0810 22:59:22.464162 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.464164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.464334 (+ 170us) server_negotiation.cc:407] Connection header received
0810 22:59:22.464911 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.464913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.464924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.464983 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:22.465917 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.466403 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.467425 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.467613 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.467670 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.469992 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:22.469994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:22.469994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:22.470001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:22.471154 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.471522 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.471524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.471524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.471602 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.471885 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.471886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.471887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.472018 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:22.472099 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.472366 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.472447 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 22:59:22.472756 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.464003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42387 (local address 127.6.47.130:46547)
0810 22:59:22.464204 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.464247 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.464813 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.465035 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.465038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.465636 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:22.465826 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.465831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.466486 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.466487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.467355 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.467357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.467689 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.467690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.467785 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.468291 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:22.468299 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:22.469911 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:22.471221 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.471226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.471228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.471424 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.471703 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.471704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.471704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.471796 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.472093 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:22.472101 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:22.472239 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.472345 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.472492 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
I20260810 22:59:22.473214 25502 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42387
I20260810 22:59:22.473309 25502 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:22.473501 25502 heartbeater.cc:511] Master 127.6.47.190:42387 requested a full tablet report, sending...
I20260810 22:59:22.473914 24923 ts_manager.cc:194] Registered new tserver with Master: 95113b8315784e3fae82f2ec7b80d62b (127.6.47.130:43931)
I20260810 22:59:22.474429 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:46547
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 22:59:22.497435 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 22:59:22.578910 25512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 22:59:22.579141 25512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 22:59:22.579190 25512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 22:59:22.580317 25512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 22:59:22.580380 25512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 22:59:22.580425 25512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 22:59:22.581967 25512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 22:59:22.582255 25512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 22:59:22.582489 25512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 22:59:22.587417 25523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 22:59:22.587476 25525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:22.587679 25512 server_base.cc:1034] running on GCE node
W20260810 22:59:22.587476 25522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 22:59:22.588060 25512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 22:59:22.588341 25512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 22:59:22.589504 25512 hybrid_clock.cc:648] HybridClock initialized: now 1786402762589478 us; error 42 us; skew 500 ppm
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 22:59:22.591948 25512 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 22:59:22.592381 25512 webserver.cc:460] Webserver started at http://127.6.47.131:41675/ using document root <none> and password file <none>
I20260810 22:59:22.592691 25512 fs_manager.cc:359] Metadata directory not provided
I20260810 22:59:22.592761 25512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 22:59:22.592904 25512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 22:59:22.593986 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "3da41106cab146eaa8d4e5cf3f26a5c7"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "2a484872f713a63b41199fa0c1c79f86"
server_key_iv: "ba696e2f645ad2136b17f98f07eb0cbc"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.594342 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "3da41106cab146eaa8d4e5cf3f26a5c7"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "2a484872f713a63b41199fa0c1c79f86"
server_key_iv: "ba696e2f645ad2136b17f98f07eb0cbc"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.595901 25512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 22:59:22.596688 25541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.596863 25512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 22:59:22.596967 25512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "3da41106cab146eaa8d4e5cf3f26a5c7"
format_stamp: "Formatted at 2026-08-10 22:59:22 on dist-test-slave-rwrg"
server_key: "2a484872f713a63b41199fa0c1c79f86"
server_key_iv: "ba696e2f645ad2136b17f98f07eb0cbc"
server_key_version: "encryptionkey@0"
I20260810 22:59:22.597049 25512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 22:59:22.614123 25512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 22:59:22.615952 25512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 22:59:22.616056 25512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 22:59:22.616225 25512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 22:59:22.616600 25512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 22:59:22.616636 25512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.616657 25512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 22:59:22.616703 25512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 22:59:22.622341 25512 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:34857
I20260810 22:59:22.622404 25736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:34857 every 8 connection(s)
I20260810 22:59:22.622756 25512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 22:59:22 dist-test-slave-rwrg krb5kdc[24837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402762, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 22:59:22.631753 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.623501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:50563 (local address 127.6.47.190:42387)
0810 22:59:22.623585 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.623587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.623783 (+ 196us) server_negotiation.cc:407] Connection header received
0810 22:59:22.624261 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.624263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.624272 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.624324 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:22.625312 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.625830 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.626787 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.626982 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.627047 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.629189 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:22.629191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:22.629191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:22.629201 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:22.630548 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.630905 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.630906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.630906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.630957 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.631177 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.631178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.631178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.631295 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:22.631347 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.631632 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.631691 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 22:59:22.631949 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.623454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42387 (local address 127.6.47.131:50563)
0810 22:59:22.623656 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.623698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.624178 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.624375 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.624378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.625008 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0810 22:59:22.625197 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.625201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.625918 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.625919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.626695 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.626697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.627067 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.627068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.627164 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.627566 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 22:59:22.627574 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 22:59:22.628982 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 22:59:22.630621 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.630627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.630628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.630840 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.631015 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 22:59:22.631016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 22:59:22.631016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 22:59:22.631124 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 22:59:22.631368 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 22:59:22.631375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 22:59:22.631513 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.631600 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.631716 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":136,"threads_started":1}
I20260810 22:59:22.632391 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 25512
I20260810 22:59:22.632485 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 22:59:22.632593 25737 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42387
I20260810 22:59:22.632715 6334 external_mini_cluster.cc:1428] Setting key 00626258dd398c116b33b58aebedb5ac
I20260810 22:59:22.632725 25737 heartbeater.cc:464] Registering TS with master...
I20260810 22:59:22.632896 25737 heartbeater.cc:511] Master 127.6.47.190:42387 requested a full tablet report, sending...
I20260810 22:59:22.633279 24923 ts_manager.cc:194] Registered new tserver with Master: 3da41106cab146eaa8d4e5cf3f26a5c7 (127.6.47.131:34857)
I20260810 22:59:22.633677 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 22:59:22.633729 24923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:50563
I20260810 22:59:22.640583 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.635088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.6.47.190:42387)
0810 22:59:22.635148 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.635150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.635278 (+ 128us) server_negotiation.cc:407] Connection header received
0810 22:59:22.635318 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.635319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.635327 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.635379 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 22:59:22.635869 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.636311 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.637043 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.637223 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.637276 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.638458 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 22:59:22.638459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 22:59:22.638459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 22:59:22.638464 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 22:59:22.639576 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.639885 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.639886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.639886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.639930 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 22:59:22.640116 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 22:59:22.640117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 22:59:22.640117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 22:59:22.640241 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 22:59:22.640311 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.640409 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.640461 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 22:59:22.642062 24923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 22:59:22.643661 24923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 22:59:22.650439 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43008 (local address 127.6.47.130:43931)
0810 22:59:22.646612 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.646616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.646807 (+ 191us) server_negotiation.cc:407] Connection header received
0810 22:59:22.646860 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.646862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.646880 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.646951 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:59:22.647186 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.647891 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.649504 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.649998 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650086 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650148 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.650315 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.650357 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260810 22:59:22.650454 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43931 (local address 127.0.0.1:43008)
0810 22:59:22.646662 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.646685 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.646740 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.647031 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.647032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.647034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:59:22.647085 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.647086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.647997 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.647998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.649387 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.649390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.650099 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.650183 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650196 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.650295 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.650351 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":71,"thread_start_us":72,"threads_started":1}
I20260810 22:59:22.650641 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.6.47.129:33371)
0810 22:59:22.646396 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.646401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.646412 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:59:22.646456 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.646458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.646476 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.646556 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:59:22.646794 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.647937 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.649330 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.650027 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650099 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650168 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.650496 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.650551 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":103,"threads_started":1}
I20260810 22:59:22.650655 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:33371 (local address 127.0.0.1:56604)
0810 22:59:22.646163 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.646194 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.646273 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.646576 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.646579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.646582 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:59:22.646724 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.646727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.648054 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.648055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.649229 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.649231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.650208 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.650286 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650290 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.650344 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.650395 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":71,"thread_start_us":72,"threads_started":1}
I20260810 22:59:22.651015 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:34857 (local address 127.0.0.1:41126)
0810 22:59:22.646667 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.646869 (+ 202us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.646920 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.647144 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.647145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.647146 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:59:22.647189 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.647191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.648335 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.648336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.649448 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.649451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.650481 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.650802 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650807 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.650897 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.650937 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":65,"threads_started":1}
I20260810 22:59:22.651156 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.646524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.6.47.131:34857)
0810 22:59:22.646728 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.646732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.646977 (+ 245us) server_negotiation.cc:407] Connection header received
0810 22:59:22.647023 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.647025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.647038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.647125 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:59:22.647245 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.648235 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.649527 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.650337 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.650394 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.650446 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.651010 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.651055 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":113,"threads_started":1}
I20260810 22:59:22.651372 25392 tablet_service.cc:1467] Processing CreateTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 (DEFAULT_TABLE table=test-table [id=9cf9b04d66c5453896d7960c2cabd781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:59:22.651372 25152 tablet_service.cc:1467] Processing CreateTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 (DEFAULT_TABLE table=test-table [id=9cf9b04d66c5453896d7960c2cabd781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:59:22.651733 25392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67eef4395d9340f98c7c68ff987c5fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:22.651733 25152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67eef4395d9340f98c7c68ff987c5fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:22.653191 25618 tablet_service.cc:1467] Processing CreateTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 (DEFAULT_TABLE table=test-table [id=9cf9b04d66c5453896d7960c2cabd781]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 22:59:22.653492 25618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67eef4395d9340f98c7c68ff987c5fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 22:59:22.655368 25763 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Bootstrap starting.
I20260810 22:59:22.656082 25763 tablet_bootstrap.cc:654] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:22.656337 25766 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Bootstrap starting.
I20260810 22:59:22.656718 25763 log.cc:824] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:22.657100 25766 tablet_bootstrap.cc:654] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:22.657121 25765 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Bootstrap starting.
I20260810 22:59:22.657635 25766 log.cc:824] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:22.657691 25765 tablet_bootstrap.cc:654] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Neither blocks nor log segments found. Creating new log.
I20260810 22:59:22.658088 25765 log.cc:824] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Log is configured to *not* fsync() on all Append() calls
I20260810 22:59:22.658383 25766 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: No bootstrap required, opened a new log
I20260810 22:59:22.658469 25766 ts_tablet_manager.cc:1397] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 22:59:22.658635 25763 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: No bootstrap required, opened a new log
I20260810 22:59:22.658692 25763 ts_tablet_manager.cc:1397] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 22:59:22.660056 25766 raft_consensus.cc:348] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.660252 25766 raft_consensus.cc:374] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:22.660305 25766 raft_consensus.cc:729] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da41106cab146eaa8d4e5cf3f26a5c7, State: Initialized, Role: FOLLOWER
I20260810 22:59:22.660259 25763 raft_consensus.cc:348] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.660463 25763 raft_consensus.cc:374] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:22.660529 25763 raft_consensus.cc:729] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95113b8315784e3fae82f2ec7b80d62b, State: Initialized, Role: FOLLOWER
I20260810 22:59:22.660463 25766 consensus_queue.cc:260] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.660655 25763 consensus_queue.cc:260] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.660784 25737 heartbeater.cc:503] Master 127.6.47.190:42387 was elected leader, sending a full tablet report...
I20260810 22:59:22.660782 25766 ts_tablet_manager.cc:1428] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 22:59:22.660961 25763 ts_tablet_manager.cc:1428] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:59:22.660984 25502 heartbeater.cc:503] Master 127.6.47.190:42387 was elected leader, sending a full tablet report...
I20260810 22:59:22.661370 25765 tablet_bootstrap.cc:492] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: No bootstrap required, opened a new log
I20260810 22:59:22.661456 25765 ts_tablet_manager.cc:1397] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 22:59:22.663084 25765 raft_consensus.cc:348] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.663223 25765 raft_consensus.cc:374] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 22:59:22.663269 25765 raft_consensus.cc:729] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 392f8c2991a842388bd0fa4430b94560, State: Initialized, Role: FOLLOWER
I20260810 22:59:22.663375 25765 consensus_queue.cc:260] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.663657 25765 ts_tablet_manager.cc:1428] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 22:59:22.663697 25276 heartbeater.cc:503] Master 127.6.47.190:42387 was elected leader, sending a full tablet report...
W20260810 22:59:22.713946 25503 tablet.cc:2369] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:59:22.791049 25280 tablet.cc:2369] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 22:59:22.873406 25738 tablet.cc:2369] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 22:59:22.913457 25778 raft_consensus.cc:482] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 22:59:22.913640 25778 raft_consensus.cc:504] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.914043 25778 leader_election.cc:288] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95113b8315784e3fae82f2ec7b80d62b (127.6.47.130:43931), 3da41106cab146eaa8d4e5cf3f26a5c7 (127.6.47.131:34857)
I20260810 22:59:22.917114 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.914293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43931 (local address 127.6.47.129:55523)
0810 22:59:22.914447 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.914461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.914549 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.914762 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.914764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.914767 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:59:22.914848 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.914849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.915489 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.915490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.916246 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.916248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.916846 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.916847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.916922 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.916926 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.916993 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.917031 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260810 22:59:22.917100 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.914372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55523 (local address 127.6.47.130:43931)
0810 22:59:22.914438 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.914446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.914539 (+ 93us) server_negotiation.cc:407] Connection header received
0810 22:59:22.914639 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.914640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.914654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.914733 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:59:22.914922 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.915402 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.916348 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.916763 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.916846 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.916855 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.916999 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.917035 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:59:22.917477 25429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67eef4395d9340f98c7c68ff987c5fe9" candidate_uuid: "392f8c2991a842388bd0fa4430b94560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95113b8315784e3fae82f2ec7b80d62b" is_pre_election: true
I20260810 22:59:22.917657 25429 raft_consensus.cc:2454] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 392f8c2991a842388bd0fa4430b94560 in term 0.
I20260810 22:59:22.917796 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.914676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:36109 (local address 127.6.47.131:34857)
0810 22:59:22.914749 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.914752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.914764 (+ 12us) server_negotiation.cc:407] Connection header received
0810 22:59:22.914864 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.914866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.914884 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.914964 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 22:59:22.915163 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.915675 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.916825 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.917377 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.917477 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.917486 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.917685 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.917728 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 22:59:22.917829 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.914565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:34857 (local address 127.6.47.129:36109)
0810 22:59:22.914692 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.914704 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 22:59:22.914754 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 22:59:22.914968 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 22:59:22.914969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 22:59:22.914971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 22:59:22.915070 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.915072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.915785 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.915786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.916725 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 22:59:22.916728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.917478 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.917480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 22:59:22.917570 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.917574 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 22:59:22.917662 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 22:59:22.917705 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260810 22:59:22.917968 25065 leader_election.cc:302] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 392f8c2991a842388bd0fa4430b94560, 95113b8315784e3fae82f2ec7b80d62b; no voters:
I20260810 22:59:22.918120 25778 raft_consensus.cc:2781] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 22:59:22.918200 25778 raft_consensus.cc:482] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 22:59:22.918290 25778 raft_consensus.cc:3037] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:22.918515 25658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67eef4395d9340f98c7c68ff987c5fe9" candidate_uuid: "392f8c2991a842388bd0fa4430b94560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" is_pre_election: true
I20260810 22:59:22.918694 25658 raft_consensus.cc:2454] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 392f8c2991a842388bd0fa4430b94560 in term 0.
I20260810 22:59:22.919339 25778 raft_consensus.cc:504] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.919500 25778 leader_election.cc:288] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [CANDIDATE]: Term 1 election: Requested vote from peers 95113b8315784e3fae82f2ec7b80d62b (127.6.47.130:43931), 3da41106cab146eaa8d4e5cf3f26a5c7 (127.6.47.131:34857)
I20260810 22:59:22.919656 25429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67eef4395d9340f98c7c68ff987c5fe9" candidate_uuid: "392f8c2991a842388bd0fa4430b94560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95113b8315784e3fae82f2ec7b80d62b"
I20260810 22:59:22.919706 25658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67eef4395d9340f98c7c68ff987c5fe9" candidate_uuid: "392f8c2991a842388bd0fa4430b94560" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7"
I20260810 22:59:22.919759 25429 raft_consensus.cc:3037] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:22.919826 25658 raft_consensus.cc:3037] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 22:59:22.920522 25429 raft_consensus.cc:2454] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 392f8c2991a842388bd0fa4430b94560 in term 1.
I20260810 22:59:22.920697 25065 leader_election.cc:302] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 392f8c2991a842388bd0fa4430b94560, 95113b8315784e3fae82f2ec7b80d62b; no voters:
I20260810 22:59:22.920782 25778 raft_consensus.cc:2781] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 22:59:22.920825 25658 raft_consensus.cc:2454] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 392f8c2991a842388bd0fa4430b94560 in term 1.
I20260810 22:59:22.920933 25778 raft_consensus.cc:686] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [term 1 LEADER]: Becoming Leader. State: Replica: 392f8c2991a842388bd0fa4430b94560, State: Running, Role: LEADER
I20260810 22:59:22.921028 25778 consensus_queue.cc:237] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } }
I20260810 22:59:22.921937 24923 catalog_manager.cc:5495] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 reported cstate change: term changed from 0 to 1, leader changed from <none> to 392f8c2991a842388bd0fa4430b94560 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "392f8c2991a842388bd0fa4430b94560" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "392f8c2991a842388bd0fa4430b94560" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 } health_report { overall_health: UNKNOWN } } }
I20260810 22:59:22.935302 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.932300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56618 (local address 127.6.47.129:33371)
0810 22:59:22.932364 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 22:59:22.932367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 22:59:22.932378 (+ 11us) server_negotiation.cc:407] Connection header received
0810 22:59:22.932583 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 22:59:22.932584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 22:59:22.932604 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 22:59:22.932697 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 22:59:22.932869 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.933376 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.934261 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 22:59:22.934598 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 22:59:22.934697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 22:59:22.934912 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 22:59:22.935038 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 22:59:22.935152 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 22:59:22.935188 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 22:59:22.935211 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 22:59:22.937064 25658 raft_consensus.cc:1261] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 1 FOLLOWER]: Refusing update from remote peer 392f8c2991a842388bd0fa4430b94560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:59:22.937098 25429 raft_consensus.cc:1261] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 1 FOLLOWER]: Refusing update from remote peer 392f8c2991a842388bd0fa4430b94560: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 22:59:22.937474 25784 consensus_queue.cc:1035] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3da41106cab146eaa8d4e5cf3f26a5c7" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:59:22.937616 25778 consensus_queue.cc:1035] T 67eef4395d9340f98c7c68ff987c5fe9 P 392f8c2991a842388bd0fa4430b94560 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95113b8315784e3fae82f2ec7b80d62b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 22:59:22.940017 25788 mvcc.cc:205] Tried to move back new op lower bound from 7317105716987650048 to 7317105716925100032. Current Snapshot: MvccSnapshot[applied={T|T < 7317105716987650048}]
I20260810 22:59:22.941195 25789 mvcc.cc:205] Tried to move back new op lower bound from 7317105716987650048 to 7317105716925100032. Current Snapshot: MvccSnapshot[applied={T|T < 7317105716987650048 or (T in {7317105716987650048})}]
I20260810 22:59:22.942395 24921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:59:22.942497 24921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 22:59:22.943297 24921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2aa1571c74b94256945e28a39fa40093: Sending DeleteTablet for 3 replicas of tablet 67eef4395d9340f98c7c68ff987c5fe9
I20260810 22:59:22.943600 25152 tablet_service.cc:1514] Processing DeleteTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:59:22 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:56604
I20260810 22:59:22.943717 25392 tablet_service.cc:1514] Processing DeleteTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:59:22 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43008
I20260810 22:59:22.943727 25618 tablet_service.cc:1514] Processing DeleteTablet for tablet 67eef4395d9340f98c7c68ff987c5fe9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 22:59:22 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:41126
I20260810 22:59:22.943976 25812 tablet_replica.cc:330] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: stopping tablet replica
I20260810 22:59:22.944000 25810 tablet_replica.cc:330] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: stopping tablet replica
I20260810 22:59:22.944109 25810 raft_consensus.cc:2229] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:59:22.944139 25812 raft_consensus.cc:2229] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 22:59:22.944267 25812 raft_consensus.cc:2258] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:59:22.944267 25810 raft_consensus.cc:2258] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 22:59:22.944273 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 25038
I20260810 22:59:22.944789 25812 ts_tablet_manager.cc:1905] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:59:22.946539 25812 ts_tablet_manager.cc:1918] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:59:22.946630 25812 log.cc:1196] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/67eef4395d9340f98c7c68ff987c5fe9
I20260810 22:59:22.946889 25812 ts_tablet_manager.cc:1939] T 67eef4395d9340f98c7c68ff987c5fe9 P 3da41106cab146eaa8d4e5cf3f26a5c7: Deleting consensus metadata
I20260810 22:59:22.947782 24892 catalog_manager.cc:4841] TS 3da41106cab146eaa8d4e5cf3f26a5c7 (127.6.47.131:34857): tablet 67eef4395d9340f98c7c68ff987c5fe9 (table test-table [id=9cf9b04d66c5453896d7960c2cabd781]) successfully deleted
I20260810 22:59:22.948251 25810 ts_tablet_manager.cc:1905] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 22:59:22.950770 25810 ts_tablet_manager.cc:1918] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 22:59:22.950862 25810 log.cc:1196] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/67eef4395d9340f98c7c68ff987c5fe9
I20260810 22:59:22.951118 25810 ts_tablet_manager.cc:1939] T 67eef4395d9340f98c7c68ff987c5fe9 P 95113b8315784e3fae82f2ec7b80d62b: Deleting consensus metadata
I20260810 22:59:22.951543 24892 catalog_manager.cc:4841] TS 95113b8315784e3fae82f2ec7b80d62b (127.6.47.130:43931): tablet 67eef4395d9340f98c7c68ff987c5fe9 (table test-table [id=9cf9b04d66c5453896d7960c2cabd781]) successfully deleted
W20260810 22:59:22.960925 24892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:33371 (error 108)
I20260810 22:59:22.961203 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 25290
I20260810 22:59:22.965639 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 22:59:22.965458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:33371 (local address 127.0.0.1:56628)
0810 22:59:22.965524 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 22:59:22.965590 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 22:59:22.965903 24892 catalog_manager.cc:4584] TS 392f8c2991a842388bd0fa4430b94560 (127.6.47.129:33371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67eef4395d9340f98c7c68ff987c5fe9: Network error: Client connection negotiation failed: client connection to 127.6.47.129:33371: connect: Connection refused (error 111)
I20260810 22:59:22.971704 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 25512
I20260810 22:59:22.995270 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 24861
2026-08-10T22:59:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2445 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 22:59:23 dist-test-slave-rwrg krb5kdc[25818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 22:59:23 dist-test-slave-rwrg krb5kdc[25818](info): set up 2 sockets
Aug 10 22:59:23 dist-test-slave-rwrg krb5kdc[25818](info): commencing operation
krb5kdc: starting...
W20260810 22:59:24.290143 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 22:59:24 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T22:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T22:59:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangeradmin_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangeradmin_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangerlookup_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangerlookup_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
I20260810 22:59:24.461021 6334 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfP10h8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/postgres -l logfile start
2026-08-10 22:59:26.890 UTC [25870] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 22:59:26.890 UTC [25870] LOG: listening on IPv4 address "127.6.47.148", port 59445
2026-08-10 22:59:26.894 UTC [25870] LOG: listening on Unix socket "/tmp/.s.PGSQL.59445"
2026-08-10 22:59:26.929 UTC [25873] LOG: database system was shut down at 2026-08-10 22:59:25 UTC
2026-08-10 22:59:26.934 UTC [25870] LOG: database system is ready to accept connections
I20260810 22:59:28.114487 6334 mini_postgres.cc:96] Postgres bound to 59445
2026-08-10 22:59:28.121 UTC [25884] FATAL: database "slave" does not exist
127.6.47.148:59445 - accepting connections
I20260810 22:59:28.122447 6334 mini_ranger.cc:161] Starting Ranger...
I20260810 22:59:28.138276 6334 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 22:59:28.580256 6334 mini_ranger.cc:87] Created ranger Postgres database
I20260810 22:59:28.580345 6334 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin
2026-08-10 22:59:28,869 [I] DB FLAVOR :POSTGRES
2026-08-10 22:59:28,869 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:59:28,869 [I] Checking connection..
2026-08-10 22:59:28,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 22:59:29,131 [I] Checking connection passed.
2026-08-10 22:59:29,132 [I] --------- Verifying version history table ---------
2026-08-10 22:59:29,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:59:29,401 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:59:29,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:59:29,668 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 22:59:29,668 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 22:59:29,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 22:59:29.937 UTC [25993] WARNING: there is no transaction in progress
2026-08-10 22:59:29,947 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 22:59:29,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 22:59:30,208 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 22:59:30,208 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:59:30,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 22:59:30,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 22:59:30,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rwrg.c.gcp-upstream.internal','N') ;"
2026-08-10 22:59:30,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:59:31,239 [I] Table x_portal_user does not exist in database ranger
2026-08-10 22:59:31,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:59:31,512 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 22:59:31,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:59:31,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:59:32,025 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 22:59:32,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 22:59:32.963 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:32.978 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:32.994 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.008 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.024 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.044 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.104 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.111 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.124 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.134 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.145 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.159 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.169 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.175 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.181 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33.187 UTC [26183] WARNING: there is no transaction in progress
2026-08-10 22:59:33,801 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 22:59:33,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 22:59:34,042 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 22:59:34,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 22:59:34,312 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 22:59:34,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 22:59:34,576 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 22:59:34,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 22:59:34,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 22:59:35,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rwrg.c.gcp-upstream.internal';"
2026-08-10 22:59:35,377 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 22:59:35,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 22:59:35,633 [I] DB_PATCHES have already been applied
I20260810 22:59:35.638861 6334 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfP10h8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 22:59:35.638931 6334 mini_ranger.cc:193] Using host: 127.6.47.148
I20260810 22:59:35.643010 6334 mini_ranger.cc:227] Ranger admin URL: http://127.6.47.148:39567
Aug 10, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 10:59:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 10:59:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 10:59:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 10:59:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 10:59:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 10:59:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 22:59:37.071076 6334 mini_ranger.cc:160] Time spent starting Ranger: real 8.949s user 0.007s sys 0.005s
Aug 10, 2026 10:59:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 10:59:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 10:59:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 10:59:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 10:59:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 22:59:57.733080 6334 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangerkms_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/rangerkms_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 22:59:57 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402797, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 22:59:57.816313 6334 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 22:59:57.826329 6334 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 22:59:58.650573 6334 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 22:59:58.650664 6334 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms
I20260810 22:59:58.650691 6334 mini_ranger_kms.cc:227] Using postgres at 127.6.47.148:59445
2026-08-10 22:59:59,258 [I] DB FLAVOR :POSTGRES
2026-08-10 22:59:59,259 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 22:59:59,259 [I] Checking connection
2026-08-10 22:59:59,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 22:59:59,525 [I] connection success
2026-08-10 22:59:59,525 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 22:59:59,525 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 22:59:59,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 22:59:59,810 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 22:59:59,811 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 22:59:59,811 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 22:59:59,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:59445/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 23:00:00,101 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 23:00:00.575629 6334 mini_ranger_kms.cc:310] Created kms service
I20260810 23:00:00.752939 6334 mini_ranger_kms.cc:326] Created kudu user
I20260810 23:00:00.800319 6334 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 23:00:00 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402797, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:01.207389 6334 mini_ranger_kms.cc:384] Added ranger policy
I20260810 23:00:01.207511 6334 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 23:00:01.207537 6334 mini_ranger_kms.cc:240] Using host: 127.6.47.148
I20260810 23:00:01.212661 6334 mini_ranger_kms.cc:276] Ranger KMS PID: 26488
I20260810 23:00:01.212795 6334 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.47.148:58079
Aug 10, 2026 11:00:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:00:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:00:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:00:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:00:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:00:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 23:00:04 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786402804, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:04.279160 6334 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.463s user 0.007s sys 0.278s
I20260810 23:00:04.279249 6334 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 23:00:04.279340 6334 mini_ranger_kms.cc:401] 127.6.47.148:58079/kms/v1/keys
Aug 10 23:00:04 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402804, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
Aug 10 23:00:05 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402804, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
Aug 10 23:00:07 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:00:07.499432 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42199
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42199
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58079/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:07.608973 26565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:07.609174 26565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:07.609194 26565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:07.610360 26565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:00:07.610404 26565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:07.610419 26565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:07.610431 26565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:00:07.610456 26565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:00:07.611994 26565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.47.148:58079/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42199
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:07.612246 26565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:07.612478 26565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:07.617959 26577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:07.617930 26573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:07.617936 26565 server_base.cc:1034] running on GCE node
W20260810 23:00:07.618115 26574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:07.618721 26565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:07.619019 26565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:07.620185 26565 hybrid_clock.cc:648] HybridClock initialized: now 1786402807620172 us; error 31 us; skew 500 ppm
Aug 10 23:00:07 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402807, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:07.622782 26565 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:00:07.623307 26565 webserver.cc:460] Webserver started at http://127.6.47.190:43563/ using document root <none> and password file <none>
I20260810 23:00:07.623530 26565 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:07.623569 26565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:07.623708 26565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:07 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402807, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:07.670917 26565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd"
format_stamp: "Formatted at 2026-08-10 23:00:07 on dist-test-slave-rwrg"
server_key: "646d8618a40ce5d5813b38fa995cf38f"
server_key_iv: "ac1a103828ec57370ac3eddb24d2c6ac"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:07.671407 26565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd"
format_stamp: "Formatted at 2026-08-10 23:00:07 on dist-test-slave-rwrg"
server_key: "646d8618a40ce5d5813b38fa995cf38f"
server_key_iv: "ac1a103828ec57370ac3eddb24d2c6ac"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:07.673005 26565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:00:07.688732 26588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:07.688963 26565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:07.689098 26565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd"
format_stamp: "Formatted at 2026-08-10 23:00:07 on dist-test-slave-rwrg"
server_key: "646d8618a40ce5d5813b38fa995cf38f"
server_key_iv: "ac1a103828ec57370ac3eddb24d2c6ac"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:07.689225 26565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:07.703292 26565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:07.705978 26565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:07.706197 26565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:07.717679 26565 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42199
I20260810 23:00:07.717772 26675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42199 every 8 connection(s)
I20260810 23:00:07.718209 26565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:00:07.719172 26677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:00:07.722352 26677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: Bootstrap starting.
I20260810 23:00:07.723492 26677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: Neither blocks nor log segments found. Creating new log.
I20260810 23:00:07.724220 26677 log.cc:824] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:07.725028 26677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: No bootstrap required, opened a new log
I20260810 23:00:07.726990 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 26565
I20260810 23:00:07.727131 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:00:07.727406 6334 external_mini_cluster.cc:1428] Setting key 4e47ac328e26cfffab1112d0b376d9a5
I20260810 23:00:07.728940 26677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } }
I20260810 23:00:07.729142 26677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:00:07.729188 26677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d0c0e29aea84e949b7bfcaafecdd1bd, State: Initialized, Role: FOLLOWER
I20260810 23:00:07.729512 26677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } }
I20260810 23:00:07.729709 26677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:00:07.729773 26688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:00:07.729928 26688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } }
I20260810 23:00:07.730106 26677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:00:07.730111 26688 leader_election.cc:302] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d0c0e29aea84e949b7bfcaafecdd1bd; no voters:
I20260810 23:00:07.730230 26677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:00:07.730453 26688 leader_election.cc:288] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260810 23:00:07.731432 26677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } }
I20260810 23:00:07.731523 26677 leader_election.cc:302] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d0c0e29aea84e949b7bfcaafecdd1bd; no voters:
I20260810 23:00:07.731559 26677 leader_election.cc:288] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:00:07.731621 26688 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 23:00:07.731866 26677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:00:07.731985 26688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:00:07.732033 26688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [term 1 LEADER]: Becoming Leader. State: Replica: 5d0c0e29aea84e949b7bfcaafecdd1bd, State: Running, Role: LEADER
I20260810 23:00:07.732100 26688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } }
I20260810 23:00:07.732569 26688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } } }
I20260810 23:00:07.732604 26689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d0c0e29aea84e949b7bfcaafecdd1bd. Latest consensus state: current_term: 1 leader_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d0c0e29aea84e949b7bfcaafecdd1bd" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42199 } } }
I20260810 23:00:07.732661 26688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:07.732897 26689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:07.733117 26696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:00:07.733935 26696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:00:07.734362 26677 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 23:00:07.735865 26714 mvcc.cc:205] Tried to move back new op lower bound from 7317105900483358720 to 7317105900471050240. Current Snapshot: MvccSnapshot[applied={T|T < 7317105900483358720}]
I20260810 23:00:07.736315 26696 catalog_manager.cc:1347] Generated new cluster ID: 0e91699219c4476f89d1d1cbf607f030
I20260810 23:00:07.736370 26696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:00:07 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:07.742434 26687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:07.728021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.6.47.190:42199)
0810 23:00:07.728366 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:07.728378 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:07.728436 (+ 58us) server_negotiation.cc:407] Connection header received
0810 23:00:07.728924 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:07.728934 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:07.729033 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:07.729208 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:07.730343 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:07.732214 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:07.733509 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:07.734225 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:07.734346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:07.737579 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:07.737583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:07.737583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:07.737595 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:07.740077 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:07.740487 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:07.740490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:07.740490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:07.740690 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:07.740958 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:07.740959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:07.740959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:07.741318 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:07.741533 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:07.742020 (+ 487us) server_negotiation.cc:299] Negotiation successful
0810 23:00:07.742144 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":130,"threads_started":1}
I20260810 23:00:07.756738 26696 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:00:07.757649 26696 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:00:07.769800 26696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: Generated new TSK 0
I20260810 23:00:07.769987 26696 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:00:08.683660 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:08.829625 26750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:08.829844 26750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:08.829869 26750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:08.831351 26750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:08.831422 26750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:08.831458 26750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:00:08.833299 26750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.47.148:58079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:08.833604 26750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:08.833886 26750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:08.840854 26755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:08.840901 26756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:08.840950 26758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:08.841857 26750 server_base.cc:1034] running on GCE node
I20260810 23:00:08.842106 26750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:08.842370 26750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:08.843538 26750 hybrid_clock.cc:648] HybridClock initialized: now 1786402808843530 us; error 48 us; skew 500 ppm
Aug 10 23:00:08 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402808, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:08.847021 26750 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:00:08.847563 26750 webserver.cc:460] Webserver started at http://127.6.47.129:37731/ using document root <none> and password file <none>
I20260810 23:00:08.847848 26750 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:08.847895 26750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:08.848032 26750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:08 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402808, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:08.863420 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "ff49ef7897904f74ad39b854a83f00e3"
format_stamp: "Formatted at 2026-08-10 23:00:08 on dist-test-slave-rwrg"
server_key: "8e1dabcb525e9cd716ec4bdb6c954eea"
server_key_iv: "7300e37ce66e47c0ffe541cff91ac202"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:08.865271 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "ff49ef7897904f74ad39b854a83f00e3"
format_stamp: "Formatted at 2026-08-10 23:00:08 on dist-test-slave-rwrg"
server_key: "8e1dabcb525e9cd716ec4bdb6c954eea"
server_key_iv: "7300e37ce66e47c0ffe541cff91ac202"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:08.867686 26750 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260810 23:00:08.891374 26767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:08.891592 26750 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260810 23:00:08.891712 26750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "ff49ef7897904f74ad39b854a83f00e3"
format_stamp: "Formatted at 2026-08-10 23:00:08 on dist-test-slave-rwrg"
server_key: "8e1dabcb525e9cd716ec4bdb6c954eea"
server_key_iv: "7300e37ce66e47c0ffe541cff91ac202"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:08.891834 26750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:08.920259 26750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:08.923281 26750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:08.923599 26750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:08.924145 26750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:08.924723 26750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:08.924911 26750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:08.925009 26750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:08.925069 26750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:08.945669 26750 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:38695
I20260810 23:00:08.946038 26943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:38695 every 8 connection(s)
I20260810 23:00:08.946303 26750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:00:08.947875 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 26750
I20260810 23:00:08.947988 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:00:08.948220 6334 external_mini_cluster.cc:1428] Setting key a43781e17874b6fd3cc661f146bf64c0
Aug 10 23:00:08 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402808, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
I20260810 23:00:08.966403 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:08.947700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42199 (local address 127.6.47.129:45011)
0810 23:00:08.948937 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:08.949007 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:08.952995 (+ 3988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:08.953467 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:08.953472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:08.953991 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:08.954461 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:08.954469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:08.955687 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:08.955688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:08.957313 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:08.957316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:08.957820 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:08.957821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:08.958076 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:08.958817 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:08.958826 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:08.961309 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:08.964257 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:08.964264 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:08.964267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:08.964668 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:08.964970 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:08.964971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:08.964971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:08.965147 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:08.965618 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:08.965626 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:08.965809 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 23:00:08.965929 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 23:00:08.966103 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":212,"threads_started":1}
I20260810 23:00:08.966792 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:08.948428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:45011 (local address 127.6.47.190:42199)
0810 23:00:08.948782 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:08.948785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:08.951893 (+ 3108us) server_negotiation.cc:407] Connection header received
0810 23:00:08.953163 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:08.953164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:08.953180 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:08.953236 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:08.954616 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:08.955532 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:08.957474 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:08.957709 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:08.957795 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:08.961442 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:08.961444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:08.961445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:08.961453 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:08.964123 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:08.964778 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:08.964780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:08.964780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:08.964858 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:08.965288 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:08.965290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:08.965290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:08.965480 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:08.965628 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:08.966004 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 23:00:08.966110 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":90,"threads_started":1}
I20260810 23:00:08.967298 26944 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42199
I20260810 23:00:08.967423 26944 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:08.967638 26944 heartbeater.cc:511] Master 127.6.47.190:42199 requested a full tablet report, sending...
I20260810 23:00:08.968266 26621 ts_manager.cc:194] Registered new tserver with Master: ff49ef7897904f74ad39b854a83f00e3 (127.6.47.129:38695)
I20260810 23:00:08.969408 26621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:45011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:00:08.991703 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:09.159626 26957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:09.159891 26957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:09.159935 26957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:09.161818 26957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:09.161906 26957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:09.161954 26957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:00:09.164287 26957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.47.148:58079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:09.164608 26957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:09.164932 26957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:09.173194 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:09.173467 26970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:09.173506 26969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:09.174883 26957 server_base.cc:1034] running on GCE node
I20260810 23:00:09.175228 26957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:09.175624 26957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:09.178923 26957 hybrid_clock.cc:648] HybridClock initialized: now 1786402809178899 us; error 61 us; skew 500 ppm
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:09.182303 26957 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:00:09.182933 26957 webserver.cc:460] Webserver started at http://127.6.47.130:39667/ using document root <none> and password file <none>
I20260810 23:00:09.183302 26957 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:09.183373 26957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:09.183506 26957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:09.203418 26957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "0165266484b74519a29fe95d639363e3"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "d00d2d8f4b36ee5555bdfa7895ca273e"
server_key_iv: "6d649f75b756a9b84c5e9a606d19884c"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.206660 26957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "0165266484b74519a29fe95d639363e3"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "d00d2d8f4b36ee5555bdfa7895ca273e"
server_key_iv: "6d649f75b756a9b84c5e9a606d19884c"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.209786 26957 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260810 23:00:09.225773 26983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.225979 26957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 23:00:09.226082 26957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "0165266484b74519a29fe95d639363e3"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "d00d2d8f4b36ee5555bdfa7895ca273e"
server_key_iv: "6d649f75b756a9b84c5e9a606d19884c"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.226203 26957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:09.243242 26957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:09.246179 26957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:09.246348 26957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:09.246762 26957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:09.247253 26957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:09.247326 26957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.247411 26957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:09.247443 26957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.268154 26957 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:35431
I20260810 23:00:09.269550 26957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:00:09.271199 27169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:35431 every 8 connection(s)
I20260810 23:00:09.275494 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 26957
I20260810 23:00:09.275624 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:00:09.275852 6334 external_mini_cluster.cc:1428] Setting key fa2707a5611cc47f7f97d052bfe00d14
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:09.285794 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.272095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:54777 (local address 127.6.47.190:42199)
0810 23:00:09.272161 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.272164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.272666 (+ 502us) server_negotiation.cc:407] Connection header received
0810 23:00:09.273559 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.273561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.273577 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.273641 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:09.274703 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.275565 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.277331 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.277787 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.277840 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.281587 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:09.281588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:09.281589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:09.281598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:09.283772 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.284364 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.284365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.284366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.284430 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.284800 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.284801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.284802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.284964 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:09.285040 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.285475 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.285609 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 23:00:09.286015 27173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.272103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42199 (local address 127.6.47.130:54777)
0810 23:00:09.272454 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.272533 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.273425 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.273722 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.273726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.274093 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:09.274569 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.274577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.275708 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.275709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.277212 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.277215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.277943 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.277944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.278166 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.278965 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:09.278977 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:09.281465 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:09.283886 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:09.283894 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:09.283897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:09.284267 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:09.284522 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:09.284523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:09.284557 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:09.284715 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:09.285058 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:09.285065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:09.285253 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.285407 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.285594 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":204,"threads_started":1}
I20260810 23:00:09.286720 27170 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42199
I20260810 23:00:09.286846 27170 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:09.287068 27170 heartbeater.cc:511] Master 127.6.47.190:42199 requested a full tablet report, sending...
I20260810 23:00:09.287585 26621 ts_manager.cc:194] Registered new tserver with Master: 0165266484b74519a29fe95d639363e3 (127.6.47.130:35431)
I20260810 23:00:09.288122 26621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:54777
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:00:09.314713 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58079/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:09.451333 27180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:09.451690 27180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:09.451803 27180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:09.453121 27180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:09.453187 27180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:09.453238 27180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:00:09.455005 27180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.47.148:58079/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:09.455282 27180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:09.455518 27180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:09.462918 27187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:09.463006 27186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:09.463322 27200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:09.464258 27180 server_base.cc:1034] running on GCE node
I20260810 23:00:09.464463 27180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:09.464783 27180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:09.465953 27180 hybrid_clock.cc:648] HybridClock initialized: now 1786402809465930 us; error 38 us; skew 500 ppm
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:09.473527 27180 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:00:09.474149 27180 webserver.cc:460] Webserver started at http://127.6.47.131:45919/ using document root <none> and password file <none>
I20260810 23:00:09.474395 27180 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:09.474465 27180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:09.474612 27180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:09.488081 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "9e589ec9303046118b2e1f63b31e37dc"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "64fd2e95c1a0934566b3a5c950252c9a"
server_key_iv: "8ee47716f8ed15d49ed5ea4370c25529"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.488701 27180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "9e589ec9303046118b2e1f63b31e37dc"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "64fd2e95c1a0934566b3a5c950252c9a"
server_key_iv: "8ee47716f8ed15d49ed5ea4370c25529"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.497117 27180 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.001s
I20260810 23:00:09.511650 27211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.511883 27180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 23:00:09.512017 27180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "9e589ec9303046118b2e1f63b31e37dc"
format_stamp: "Formatted at 2026-08-10 23:00:09 on dist-test-slave-rwrg"
server_key: "64fd2e95c1a0934566b3a5c950252c9a"
server_key_iv: "8ee47716f8ed15d49ed5ea4370c25529"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:09.512171 27180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:09.533043 27180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:09.535907 27180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:09.536128 27180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:09.536636 27180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:09.537182 27180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:09.537250 27180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.537299 27180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:09.537335 27180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:09.562386 27180 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:36071
I20260810 23:00:09.562852 27394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:36071 every 8 connection(s)
I20260810 23:00:09.562933 27180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:00:09.566236 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 27180
I20260810 23:00:09.566375 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:00:09.566622 6334 external_mini_cluster.cc:1428] Setting key 4ed704bfeb8ab96f4c998fe37a0f06b0
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:09.580359 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.565348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:45629 (local address 127.6.47.190:42199)
0810 23:00:09.565776 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.565778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.568571 (+ 2793us) server_negotiation.cc:407] Connection header received
0810 23:00:09.569542 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.569544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.569556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.569588 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:09.570871 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.571388 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.572681 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.573043 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.573126 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.576957 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:09.576959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:09.576959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:09.576967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:09.578910 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.579310 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.579312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.579312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.579376 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.579668 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.579669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.579669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.579796 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:09.579869 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.580171 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.580270 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260810 23:00:09.581068 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.564976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42199 (local address 127.6.47.131:45629)
0810 23:00:09.567175 (+ 2199us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.567239 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.569407 (+ 2168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.569803 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.569808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.570310 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:09.570756 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.570762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.571483 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.571484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.572536 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.572540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.573203 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.573204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.573388 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.574190 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:09.574201 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:09.576859 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:09.578978 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:09.578986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:09.578989 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:09.579223 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:09.579444 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:09.579445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:09.579445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:09.579590 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:09.579871 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:09.579879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:09.580031 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.580138 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.580295 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2115,"thread_start_us":151,"threads_started":1}
I20260810 23:00:09.581676 27395 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42199
I20260810 23:00:09.581802 27395 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:09.582033 27395 heartbeater.cc:511] Master 127.6.47.190:42199 requested a full tablet report, sending...
I20260810 23:00:09.582607 26621 ts_manager.cc:194] Registered new tserver with Master: 9e589ec9303046118b2e1f63b31e37dc (127.6.47.131:36071)
I20260810 23:00:09.583011 26621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:45629
I20260810 23:00:09.589864 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:00:09.599640 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.591435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.6.47.190:42199)
0810 23:00:09.591479 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.591481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.591756 (+ 275us) server_negotiation.cc:407] Connection header received
0810 23:00:09.591785 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.591786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.591808 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.591882 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:09.592557 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.593435 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.594363 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.594725 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.594829 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.596319 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:09.596321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:09.596321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:09.596329 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:09.598386 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.598775 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.598776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.598777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.598844 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:09.599086 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:09.599087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:09.599087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:09.599241 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:09.599366 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.599451 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.599531 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:00:09.601688 26621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:00:09.604632 26621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:00:09.613426 27420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:36071 (local address 127.0.0.1:44910)
0810 23:00:09.608984 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.609027 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.609118 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.609400 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.609403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.609405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:00:09.609475 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.609477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.610962 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.610963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.611781 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.611783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.613082 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.613083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.613170 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.613186 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.613293 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.613352 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"thread_start_us":180,"threads_started":1}
I20260810 23:00:09.613451 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.6.47.131:36071)
0810 23:00:09.608634 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.608639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.609100 (+ 461us) server_negotiation.cc:407] Connection header received
0810 23:00:09.609266 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.609268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.609306 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.609400 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:00:09.609538 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.610886 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.612016 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.612981 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.613083 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.613254 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.613311 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.613352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
I20260810 23:00:09.614553 27274 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba281573c642a4a93b8bb13088e864 (DEFAULT_TABLE table=test-table [id=c844811f39b94de08b02aa5fa838b7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:00:09.614965 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba281573c642a4a93b8bb13088e864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:00:09.615813 27423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:38695 (local address 127.0.0.1:47156)
0810 23:00:09.609688 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.609698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.609748 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.610139 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.610139 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.610141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:00:09.610189 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.610190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.612569 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.612570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.614314 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.614317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.615535 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.615536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.615625 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.615630 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.615693 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.615737 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"thread_start_us":108,"threads_started":1}
I20260810 23:00:09.616617 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.6.47.129:38695)
0810 23:00:09.608928 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.608933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.609810 (+ 877us) server_negotiation.cc:407] Connection header received
0810 23:00:09.609852 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.609855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.609871 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.609944 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:00:09.610273 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.612421 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.614447 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.615420 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.615534 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.615602 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.616421 (+ 819us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.616468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260810 23:00:09.617653 26844 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba281573c642a4a93b8bb13088e864 (DEFAULT_TABLE table=test-table [id=c844811f39b94de08b02aa5fa838b7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:00:09.618083 26844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba281573c642a4a93b8bb13088e864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:00:09.620664 27426 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Bootstrap starting.
I20260810 23:00:09.621009 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:35431 (local address 127.0.0.1:35472)
0810 23:00:09.609277 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.609305 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.609379 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.610009 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.610010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.610012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:00:09.610048 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.610049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.612536 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.612537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.614507 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.614510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.620698 (+ 6188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.620699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.620781 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.620786 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.620881 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.620933 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":167,"threads_started":1}
I20260810 23:00:09.621071 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.608654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.6.47.130:35431)
0810 23:00:09.609040 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.609045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.609650 (+ 605us) server_negotiation.cc:407] Connection header received
0810 23:00:09.609731 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.609735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.609754 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.609882 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:00:09.610136 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.612315 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.615777 (+ 3462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.617420 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.617559 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.617732 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.620913 (+ 3181us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.620965 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":205,"threads_started":1}
I20260810 23:00:09.622136 27062 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba281573c642a4a93b8bb13088e864 (DEFAULT_TABLE table=test-table [id=c844811f39b94de08b02aa5fa838b7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:00:09.622567 27062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba281573c642a4a93b8bb13088e864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:00:09.623236 27428 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Bootstrap starting.
I20260810 23:00:09.623292 27426 tablet_bootstrap.cc:654] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Neither blocks nor log segments found. Creating new log.
I20260810 23:00:09.624706 27426 log.cc:824] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:09.624879 27428 tablet_bootstrap.cc:654] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Neither blocks nor log segments found. Creating new log.
I20260810 23:00:09.625484 27428 log.cc:824] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:09.625775 27426 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: No bootstrap required, opened a new log
I20260810 23:00:09.625849 27426 ts_tablet_manager.cc:1397] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 23:00:09.626320 27428 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: No bootstrap required, opened a new log
I20260810 23:00:09.626396 27428 ts_tablet_manager.cc:1397] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:00:09.628886 27433 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Bootstrap starting.
I20260810 23:00:09.630914 27433 tablet_bootstrap.cc:654] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Neither blocks nor log segments found. Creating new log.
I20260810 23:00:09.630923 27428 raft_consensus.cc:348] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.631119 27428 raft_consensus.cc:374] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:00:09.631166 27428 raft_consensus.cc:729] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff49ef7897904f74ad39b854a83f00e3, State: Initialized, Role: FOLLOWER
I20260810 23:00:09.631304 27428 consensus_queue.cc:260] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.631716 27428 ts_tablet_manager.cc:1428] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 23:00:09.631881 26944 heartbeater.cc:503] Master 127.6.47.190:42199 was elected leader, sending a full tablet report...
I20260810 23:00:09.631974 27433 log.cc:824] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:09.633687 27433 tablet_bootstrap.cc:492] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: No bootstrap required, opened a new log
I20260810 23:00:09.633824 27433 ts_tablet_manager.cc:1397] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 23:00:09.637068 27433 raft_consensus.cc:348] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.637264 27433 raft_consensus.cc:374] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:00:09.637315 27433 raft_consensus.cc:729] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0165266484b74519a29fe95d639363e3, State: Initialized, Role: FOLLOWER
I20260810 23:00:09.637241 27426 raft_consensus.cc:348] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.637451 27433 consensus_queue.cc:260] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.637513 27426 raft_consensus.cc:374] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:00:09.637576 27426 raft_consensus.cc:729] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e589ec9303046118b2e1f63b31e37dc, State: Initialized, Role: FOLLOWER
I20260810 23:00:09.637809 27433 ts_tablet_manager.cc:1428] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 23:00:09.637871 27170 heartbeater.cc:503] Master 127.6.47.190:42199 was elected leader, sending a full tablet report...
I20260810 23:00:09.637804 27426 consensus_queue.cc:260] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.638923 27426 ts_tablet_manager.cc:1428] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260810 23:00:09.639045 27395 heartbeater.cc:503] Master 127.6.47.190:42199 was elected leader, sending a full tablet report...
W20260810 23:00:09.698035 26947 tablet.cc:2369] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:00:09.770720 27171 tablet.cc:2369] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 23:00:09 dist-test-slave-rwrg krb5kdc[25818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402809, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
W20260810 23:00:09.816220 27397 tablet.cc:2369] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:00:09.875500 27436 raft_consensus.cc:482] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:00:09.875758 27436 raft_consensus.cc:504] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.876240 27436 leader_election.cc:288] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e589ec9303046118b2e1f63b31e37dc (127.6.47.131:36071), 0165266484b74519a29fe95d639363e3 (127.6.47.130:35431)
I20260810 23:00:09.881042 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.876636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55521 (local address 127.6.47.131:36071)
0810 23:00:09.876683 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.876685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.877413 (+ 728us) server_negotiation.cc:407] Connection header received
0810 23:00:09.877518 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.877520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.877532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.877559 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:00:09.877940 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.878678 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.880165 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.880686 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.880731 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.880741 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.880938 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.880975 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:00:09.881441 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.876719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:36071 (local address 127.6.47.129:55521)
0810 23:00:09.877324 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.877334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.877444 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.877633 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.877634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.877636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:00:09.877835 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.877837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.878816 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.878817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.880062 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.880065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.880795 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.880796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.880867 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.880870 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.881319 (+ 449us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.881363 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":159,"threads_started":1}
I20260810 23:00:09.881930 27323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba281573c642a4a93b8bb13088e864" candidate_uuid: "ff49ef7897904f74ad39b854a83f00e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e589ec9303046118b2e1f63b31e37dc" is_pre_election: true
I20260810 23:00:09.882102 27323 raft_consensus.cc:2454] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff49ef7897904f74ad39b854a83f00e3 in term 0.
I20260810 23:00:09.883090 26774 leader_election.cc:302] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e589ec9303046118b2e1f63b31e37dc, ff49ef7897904f74ad39b854a83f00e3; no voters:
I20260810 23:00:09.883234 27436 raft_consensus.cc:2781] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:00:09.883271 27436 raft_consensus.cc:482] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:00:09.883291 27436 raft_consensus.cc:3037] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:00:09.884151 27436 raft_consensus.cc:504] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.884267 27436 leader_election.cc:288] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [CANDIDATE]: Term 1 election: Requested vote from peers 9e589ec9303046118b2e1f63b31e37dc (127.6.47.131:36071), 0165266484b74519a29fe95d639363e3 (127.6.47.130:35431)
I20260810 23:00:09.884601 27323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba281573c642a4a93b8bb13088e864" candidate_uuid: "ff49ef7897904f74ad39b854a83f00e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e589ec9303046118b2e1f63b31e37dc"
I20260810 23:00:09.884704 27323 raft_consensus.cc:3037] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:00:09.885509 27323 raft_consensus.cc:2454] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff49ef7897904f74ad39b854a83f00e3 in term 1.
I20260810 23:00:09.885738 26774 leader_election.cc:302] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e589ec9303046118b2e1f63b31e37dc, ff49ef7897904f74ad39b854a83f00e3; no voters:
I20260810 23:00:09.885848 27436 raft_consensus.cc:2781] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:00:09.886166 27436 raft_consensus.cc:686] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [term 1 LEADER]: Becoming Leader. State: Replica: ff49ef7897904f74ad39b854a83f00e3, State: Running, Role: LEADER
I20260810 23:00:09.886255 27436 consensus_queue.cc:237] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } }
I20260810 23:00:09.887110 26621 catalog_manager.cc:5495] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to ff49ef7897904f74ad39b854a83f00e3 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "ff49ef7897904f74ad39b854a83f00e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff49ef7897904f74ad39b854a83f00e3" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38695 } health_report { overall_health: HEALTHY } } }
I20260810 23:00:09.889091 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.877188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:58185 (local address 127.6.47.130:35431)
0810 23:00:09.877228 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.877230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.881871 (+ 4641us) server_negotiation.cc:407] Connection header received
0810 23:00:09.881974 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.881975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.881989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.882018 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:00:09.882190 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.882854 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.888175 (+ 5321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.888746 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.888794 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.888803 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.888986 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.889024 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:00:09.889453 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.876983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:35431 (local address 127.6.47.129:58185)
0810 23:00:09.881817 (+ 4834us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.881829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:09.881899 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:09.882082 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:09.882083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:09.882084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:00:09.882112 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.882114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.886936 (+ 4822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.886936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.888062 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:09.888066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.888858 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.888859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:09.888913 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.888916 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:00:09.889348 (+ 432us) client_negotiation.cc:240] Negotiation successful
0810 23:00:09.889391 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4823,"thread_start_us":109,"threads_started":1}
I20260810 23:00:09.889890 27098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba281573c642a4a93b8bb13088e864" candidate_uuid: "ff49ef7897904f74ad39b854a83f00e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0165266484b74519a29fe95d639363e3" is_pre_election: true
I20260810 23:00:09.890025 27098 raft_consensus.cc:2454] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff49ef7897904f74ad39b854a83f00e3 in term 0.
I20260810 23:00:09.890117 27097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba281573c642a4a93b8bb13088e864" candidate_uuid: "ff49ef7897904f74ad39b854a83f00e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0165266484b74519a29fe95d639363e3"
I20260810 23:00:09.890161 27097 raft_consensus.cc:3037] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:00:09.891238 27097 raft_consensus.cc:2454] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff49ef7897904f74ad39b854a83f00e3 in term 1.
I20260810 23:00:09.902098 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.898549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.6.47.129:38695)
0810 23:00:09.898597 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:09.898598 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:09.898740 (+ 142us) server_negotiation.cc:407] Connection header received
0810 23:00:09.898779 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:09.898780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:09.898802 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:09.898868 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:00:09.899054 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.899822 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.900994 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:09.901291 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:09.901352 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:09.901561 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:00:09.901698 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:00:09.901769 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:09.901998 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 23:00:09.902022 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:00:09.904456 27097 raft_consensus.cc:1261] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 1 FOLLOWER]: Refusing update from remote peer ff49ef7897904f74ad39b854a83f00e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:00:09.904940 27436 consensus_queue.cc:1035] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0165266484b74519a29fe95d639363e3" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:00:09.907191 27323 raft_consensus.cc:1261] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 1 FOLLOWER]: Refusing update from remote peer ff49ef7897904f74ad39b854a83f00e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:00:09.907528 27436 consensus_queue.cc:1035] T a1ba281573c642a4a93b8bb13088e864 P ff49ef7897904f74ad39b854a83f00e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e589ec9303046118b2e1f63b31e37dc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:00:09.912288 27450 mvcc.cc:205] Tried to move back new op lower bound from 7317105909365596160 to 7317105909294596096. Current Snapshot: MvccSnapshot[applied={T|T < 7317105909365596160}]
I20260810 23:00:09.912637 27454 mvcc.cc:205] Tried to move back new op lower bound from 7317105909365596160 to 7317105909294596096. Current Snapshot: MvccSnapshot[applied={T|T < 7317105909365596160}]
I20260810 23:00:09.916363 27451 mvcc.cc:205] Tried to move back new op lower bound from 7317105909365596160 to 7317105909294596096. Current Snapshot: MvccSnapshot[applied={T|T < 7317105909365596160}]
I20260810 23:00:09.919046 26621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:00:09.919163 26621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:00:09.919945 26621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d0c0e29aea84e949b7bfcaafecdd1bd: Sending DeleteTablet for 3 replicas of tablet a1ba281573c642a4a93b8bb13088e864
I20260810 23:00:09.920394 27062 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba281573c642a4a93b8bb13088e864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:00:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:35472
I20260810 23:00:09.920632 27274 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba281573c642a4a93b8bb13088e864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:00:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:44910
I20260810 23:00:09.921026 26844 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba281573c642a4a93b8bb13088e864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:00:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:47156
I20260810 23:00:09.921268 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 26750
I20260810 23:00:09.930259 27470 tablet_replica.cc:330] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: stopping tablet replica
I20260810 23:00:09.930505 27470 raft_consensus.cc:2229] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:00:09.930735 27470 raft_consensus.cc:2258] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:00:09.931314 27470 ts_tablet_manager.cc:1905] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:00:09.934123 27470 ts_tablet_manager.cc:1918] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:00:09.934476 27470 log.cc:1196] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/a1ba281573c642a4a93b8bb13088e864
I20260810 23:00:09.935568 27470 ts_tablet_manager.cc:1939] T a1ba281573c642a4a93b8bb13088e864 P 9e589ec9303046118b2e1f63b31e37dc: Deleting consensus metadata
I20260810 23:00:09.936408 27467 tablet_replica.cc:330] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: stopping tablet replica
I20260810 23:00:09.936414 26594 catalog_manager.cc:4841] TS 9e589ec9303046118b2e1f63b31e37dc (127.6.47.131:36071): tablet a1ba281573c642a4a93b8bb13088e864 (table test-table [id=c844811f39b94de08b02aa5fa838b7e7]) successfully deleted
I20260810 23:00:09.936755 27467 raft_consensus.cc:2229] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:00:09.936952 27467 raft_consensus.cc:2258] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:00:09.937597 27467 ts_tablet_manager.cc:1905] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:00:09.939805 27467 ts_tablet_manager.cc:1918] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:00:09.939891 27467 log.cc:1196] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/a1ba281573c642a4a93b8bb13088e864
I20260810 23:00:09.940204 27467 ts_tablet_manager.cc:1939] T a1ba281573c642a4a93b8bb13088e864 P 0165266484b74519a29fe95d639363e3: Deleting consensus metadata
I20260810 23:00:09.940758 26594 catalog_manager.cc:4841] TS 0165266484b74519a29fe95d639363e3 (127.6.47.130:35431): tablet a1ba281573c642a4a93b8bb13088e864 (table test-table [id=c844811f39b94de08b02aa5fa838b7e7]) successfully deleted
W20260810 23:00:09.942422 26594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:38695 (error 108)
I20260810 23:00:09.943150 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 26957
I20260810 23:00:09.943167 27422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:09.943008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:38695 (local address 127.0.0.1:47176)
0810 23:00:09.943077 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:09.943125 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:00:09.948647 26594 catalog_manager.cc:4584] TS ff49ef7897904f74ad39b854a83f00e3 (127.6.47.129:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1ba281573c642a4a93b8bb13088e864: Network error: Client connection negotiation failed: client connection to 127.6.47.129:38695: connect: Connection refused (error 111)
I20260810 23:00:09.955963 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 27180
I20260810 23:00:09.989393 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 26565
I20260810 23:00:10.004330 6334 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 23:00:10.349181 6334 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 23:00:10.349320 6334 mini_ranger.cc:66] Stopping Ranger...
I20260810 23:00:10.704099 6334 mini_ranger.cc:68] Stopped Ranger
2026-08-10 23:00:10.704 UTC [25870] LOG: received smart shutdown request
2026-08-10 23:00:10.707 UTC [25870] LOG: background worker "logical replication launcher" (PID 25881) exited with exit code 1
2026-08-10 23:00:10.708 UTC [25874] LOG: shutting down
2026-08-10 23:00:10.866 UTC [25870] LOG: database system is shut down
2026-08-10T23:00:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47979 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:00:11 dist-test-slave-rwrg krb5kdc[27491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:00:11 dist-test-slave-rwrg krb5kdc[27491](info): set up 2 sockets
Aug 10 23:00:11 dist-test-slave-rwrg krb5kdc[27491](info): commencing operation
krb5kdc: starting...
W20260810 23:00:12.300485 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:00:12 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402812, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:00:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:00:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangeradmin_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangeradmin_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangerlookup_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangerlookup_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
I20260810 23:00:12.470630 6334 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfP10h8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/postgres -l logfile start
2026-08-10 23:00:14.847 UTC [27555] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 23:00:14.847 UTC [27555] LOG: listening on IPv4 address "127.6.47.148", port 46309
2026-08-10 23:00:14.851 UTC [27555] LOG: listening on Unix socket "/tmp/.s.PGSQL.46309"
2026-08-10 23:00:14.872 UTC [27558] LOG: database system was shut down at 2026-08-10 23:00:13 UTC
2026-08-10 23:00:14.875 UTC [27555] LOG: database system is ready to accept connections
I20260810 23:00:16.078794 6334 mini_postgres.cc:96] Postgres bound to 46309
2026-08-10 23:00:16.085 UTC [27571] FATAL: database "slave" does not exist
127.6.47.148:46309 - accepting connections
I20260810 23:00:16.086602 6334 mini_ranger.cc:161] Starting Ranger...
I20260810 23:00:16.101641 6334 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 23:00:16.547082 6334 mini_ranger.cc:87] Created ranger Postgres database
I20260810 23:00:16.547175 6334 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin
2026-08-10 23:00:16,835 [I] DB FLAVOR :POSTGRES
2026-08-10 23:00:16,836 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:00:16,836 [I] Checking connection..
2026-08-10 23:00:16,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 23:00:17,081 [I] Checking connection passed.
2026-08-10 23:00:17,081 [I] --------- Verifying version history table ---------
2026-08-10 23:00:17,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:00:17,351 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:00:17,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:00:17,618 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 23:00:17,619 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 23:00:17,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 23:00:17.857 UTC [27671] WARNING: there is no transaction in progress
2026-08-10 23:00:17,867 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 23:00:17,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 23:00:18,138 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 23:00:18,138 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:00:18,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 23:00:18,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 23:00:18,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rwrg.c.gcp-upstream.internal','N') ;"
2026-08-10 23:00:18,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:00:19,204 [I] Table x_portal_user does not exist in database ranger
2026-08-10 23:00:19,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:00:19,458 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 23:00:19,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:00:19,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:00:19,961 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 23:00:19,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 23:00:20.922 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:20.938 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:20.954 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:20.969 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:20.984 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.002 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.057 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.063 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.077 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.086 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.096 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.110 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.120 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.126 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.133 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21.139 UTC [27865] WARNING: there is no transaction in progress
2026-08-10 23:00:21,781 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 23:00:21,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 23:00:22,032 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 23:00:22,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 23:00:22,294 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 23:00:22,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 23:00:22,559 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 23:00:22,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 23:00:22,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 23:00:23,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rwrg.c.gcp-upstream.internal';"
2026-08-10 23:00:23,336 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 23:00:23,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 23:00:23,585 [I] DB_PATCHES have already been applied
I20260810 23:00:23.591769 6334 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfP10h8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 23:00:23.591845 6334 mini_ranger.cc:193] Using host: 127.6.47.148
I20260810 23:00:23.596236 6334 mini_ranger.cc:227] Ranger admin URL: http://127.6.47.148:40001
Aug 10, 2026 11:00:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:00:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:00:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:00:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:00:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:00:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:00:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 23:00:25.039366 6334 mini_ranger.cc:160] Time spent starting Ranger: real 8.953s user 0.000s sys 0.011s
Aug 10, 2026 11:00:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:00:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 11:00:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:00:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:00:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 23:00:47.452697 6334 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangerkms_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/rangerkms_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/HTTP_127.6.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 23:00:47 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402847, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 23:00:47.543203 6334 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 23:00:47.553931 6334 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 23:00:48.332335 6334 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 23:00:48.332409 6334 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms
I20260810 23:00:48.332422 6334 mini_ranger_kms.cc:227] Using postgres at 127.6.47.148:46309
2026-08-10 23:00:48,504 [I] DB FLAVOR :POSTGRES
2026-08-10 23:00:48,504 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 23:00:48,504 [I] Checking connection
2026-08-10 23:00:48,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 23:00:48,748 [I] connection success
2026-08-10 23:00:48,748 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 23:00:48,748 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 23:00:48,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 23:00:49,003 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 23:00:49,003 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 23:00:49,003 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 23:00:49,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.47.148:46309/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 23:00:49,287 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 23:00:49.753986 6334 mini_ranger_kms.cc:310] Created kms service
I20260810 23:00:49.975414 6334 mini_ranger_kms.cc:326] Created kudu user
I20260810 23:00:50.025374 6334 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 23:00:50 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402847, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:50.417143 6334 mini_ranger_kms.cc:384] Added ranger policy
I20260810 23:00:50.417354 6334 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfP10h8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfP10h8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 23:00:50.417383 6334 mini_ranger_kms.cc:240] Using host: 127.6.47.148
I20260810 23:00:50.421851 6334 mini_ranger_kms.cc:276] Ranger KMS PID: 28189
I20260810 23:00:50.421938 6334 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.47.148:58873
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:00:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 23:00:52 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786402852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 23:00:53 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:53.293846 6334 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.751s user 0.005s sys 0.149s
I20260810 23:00:53.293941 6334 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 23:00:53.293987 6334 mini_ranger_kms.cc:401] 127.6.47.148:58873/kms/v1/keys
Aug 10 23:00:53 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402852, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.47.148@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
Aug 10 23:00:55 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:00:55.807906 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42525
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42525
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:55.889926 28273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:55.890137 28273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:55.890159 28273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:55.891316 28273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:00:55.891359 28273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:55.891373 28273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:55.891386 28273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:00:55.891398 28273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:00:55.893061 28273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.47.148:58873/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42525
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:55.893325 28273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:55.893565 28273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:55.898949 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:55.899047 28285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:55.898993 28273 server_base.cc:1034] running on GCE node
W20260810 23:00:55.898936 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:55.899506 28273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:55.899822 28273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:55.900996 28273 hybrid_clock.cc:648] HybridClock initialized: now 1786402855900979 us; error 31 us; skew 500 ppm
Aug 10 23:00:55 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:55.904007 28273 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:00:55.904529 28273 webserver.cc:460] Webserver started at http://127.6.47.190:39337/ using document root <none> and password file <none>
I20260810 23:00:55.904773 28273 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:55.904825 28273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:55.904953 28273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:55 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402855, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:55.950335 28273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "942034e5459b40a9a3016343aca13741"
format_stamp: "Formatted at 2026-08-10 23:00:55 on dist-test-slave-rwrg"
server_key: "45dae84a3b945d7b72cdf602e3620888"
server_key_iv: "7030a820a37de9d6ae2c6339b626aedd"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:55.950798 28273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "942034e5459b40a9a3016343aca13741"
format_stamp: "Formatted at 2026-08-10 23:00:55 on dist-test-slave-rwrg"
server_key: "45dae84a3b945d7b72cdf602e3620888"
server_key_iv: "7030a820a37de9d6ae2c6339b626aedd"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:55.952385 28273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:00:55.964418 28299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:55.964726 28273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:55.964838 28273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "942034e5459b40a9a3016343aca13741"
format_stamp: "Formatted at 2026-08-10 23:00:55 on dist-test-slave-rwrg"
server_key: "45dae84a3b945d7b72cdf602e3620888"
server_key_iv: "7030a820a37de9d6ae2c6339b626aedd"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:55.964936 28273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:55.981168 28273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:55.983333 28273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:55.983515 28273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:55.988430 28273 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42525
I20260810 23:00:55.988456 28390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42525 every 8 connection(s)
I20260810 23:00:55.988865 28273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:00:55.989600 28394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:00:55.991946 28394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Bootstrap starting.
I20260810 23:00:55.992623 28394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Neither blocks nor log segments found. Creating new log.
I20260810 23:00:55.992929 28394 log.cc:824] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:55.993734 28394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: No bootstrap required, opened a new log
I20260810 23:00:55.994100 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 28273
I20260810 23:00:55.994184 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:00:55.994345 6334 external_mini_cluster.cc:1428] Setting key 6ff0c26011be775158e7dc28c94822a2
I20260810 23:00:55.996407 28394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:55.996692 28394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:00:55.996753 28394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942034e5459b40a9a3016343aca13741, State: Initialized, Role: FOLLOWER
I20260810 23:00:55.996905 28394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:55.997011 28394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:00:55.997066 28394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:00:55.997145 28394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:00:55.997709 28394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:55.997834 28394 leader_election.cc:302] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 942034e5459b40a9a3016343aca13741; no voters:
I20260810 23:00:55.999189 28394 leader_election.cc:288] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:00:55.999217 28400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:00:55.999459 28400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 LEADER]: Becoming Leader. State: Replica: 942034e5459b40a9a3016343aca13741, State: Running, Role: LEADER
I20260810 23:00:55.999591 28400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:55.999576 28394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:00:56.000020 28407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 942034e5459b40a9a3016343aca13741. Latest consensus state: current_term: 1 leader_uuid: "942034e5459b40a9a3016343aca13741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } } }
I20260810 23:00:56.000262 28407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:56.000192 28405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "942034e5459b40a9a3016343aca13741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } } }
I20260810 23:00:56.000542 28405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:56.000849 28412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:00:56.001583 28412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:00:56.001955 28394 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 23:00:56.003139 28428 mvcc.cc:205] Tried to move back new op lower bound from 7317106098186784768 to 7317106098175086592. Current Snapshot: MvccSnapshot[applied={T|T < 7317106098186784768}]
I20260810 23:00:56.004151 28412 catalog_manager.cc:1347] Generated new cluster ID: 9f316fbe4e4f473eb18f12c4344d165d
I20260810 23:00:56.004213 28412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:00:56 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:56.015780 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:55.994888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.6.47.190:42525)
0810 23:00:55.995091 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:55.995102 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:55.995224 (+ 122us) server_negotiation.cc:407] Connection header received
0810 23:00:55.996348 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:55.996359 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:55.996425 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:55.996641 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:56.000069 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:56.001578 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:56.002674 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:56.003297 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:56.003378 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:56.006597 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:56.006601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:56.006602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:56.006629 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:56.012418 (+ 5789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:56.013073 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:56.013078 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:56.013079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:56.013311 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:56.013564 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:56.013565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:56.013566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:56.014127 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:56.014354 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:56.014927 (+ 573us) server_negotiation.cc:299] Negotiation successful
0810 23:00:56.015382 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260810 23:00:56.036676 28412 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:00:56.037484 28412 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:00:56.049953 28412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Generated new TSK 0
I20260810 23:00:56.050174 28412 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:00:56.819674 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:56.929896 28460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:56.930245 28460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:56.930337 28460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:56.931767 28460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:56.931842 28460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:56.931890 28460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:00:56.934367 28460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:56.934850 28460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:56.935113 28460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:56.942458 28475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:56.943320 28470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:56.943610 28460 server_base.cc:1034] running on GCE node
W20260810 23:00:56.943780 28472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:56.944057 28460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:56.944399 28460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:56.945585 28460 hybrid_clock.cc:648] HybridClock initialized: now 1786402856945565 us; error 55 us; skew 500 ppm
Aug 10 23:00:56 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:56.949013 28460 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:00:56.949595 28460 webserver.cc:460] Webserver started at http://127.6.47.129:44153/ using document root <none> and password file <none>
I20260810 23:00:56.949896 28460 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:56.949960 28460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:56.950145 28460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:56 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:56.966086 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0"
format_stamp: "Formatted at 2026-08-10 23:00:56 on dist-test-slave-rwrg"
server_key: "4b3a5b5b1a60d9539990741eeffe15b3"
server_key_iv: "27ca770d90ee55bdedbcc244f74ebb5b"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:56.966687 28460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0"
format_stamp: "Formatted at 2026-08-10 23:00:56 on dist-test-slave-rwrg"
server_key: "4b3a5b5b1a60d9539990741eeffe15b3"
server_key_iv: "27ca770d90ee55bdedbcc244f74ebb5b"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:56.969406 28460 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 23:00:56.985569 28487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:56.985858 28460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:00:56.985985 28460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0"
format_stamp: "Formatted at 2026-08-10 23:00:56 on dist-test-slave-rwrg"
server_key: "4b3a5b5b1a60d9539990741eeffe15b3"
server_key_iv: "27ca770d90ee55bdedbcc244f74ebb5b"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:56.986096 28460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:57.005137 28460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:57.007769 28460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:57.007972 28460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:57.008258 28460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:57.008639 28460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:57.008729 28460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.008772 28460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:57.008819 28460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.016039 28460 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:45319
I20260810 23:00:57.016567 28460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:00:57.016707 28690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:45319 every 8 connection(s)
I20260810 23:00:57.022855 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 28460
I20260810 23:00:57.022971 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:00:57.023168 6334 external_mini_cluster.cc:1428] Setting key 61107171304af379b3ba5e34c5d43f99
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402856, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:57.031916 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.019652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.129:42127)
0810 23:00:57.020152 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:57.020203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:57.020970 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:57.021372 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:57.021376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:57.021633 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:57.021917 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.021924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.022985 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.022987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.024814 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.024819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.025439 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.025440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.025583 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.026249 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:57.026261 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:57.028000 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:57.030161 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.030168 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.030170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.030477 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.030706 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.030706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.030707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.030840 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.031149 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:57.031157 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:57.031359 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 23:00:57.031479 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 23:00:57.031649 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":82,"threads_started":1}
I20260810 23:00:57.032546 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.019957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:42127 (local address 127.6.47.190:42525)
0810 23:00:57.020578 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:57.020580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:57.020592 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:00:57.021160 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:57.021161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:57.021175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:57.021239 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:57.022036 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.022864 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.024915 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.025280 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.025343 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.028132 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:57.028133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:57.028134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:57.028141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:57.030044 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.030568 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.030569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.030570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.030637 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.030911 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.030912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.030912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.031047 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:57.031120 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:57.032320 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0810 23:00:57.032414 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":77,"threads_started":1}
I20260810 23:00:57.033336 28691 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:57.033442 28691 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:57.033644 28691 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:57.034308 28329 ts_manager.cc:194] Registered new tserver with Master: 1b6e99724c944aaaaedc1d0c9ccea6b0 (127.6.47.129:45319)
I20260810 23:00:57.035205 28329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:42127
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:00:57.065814 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 23:00:57.184175 28705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:57.184392 28705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:57.184417 28705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:57.186094 28705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:57.186156 28705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:57.186196 28705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:00:57.188313 28705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:57.188585 28705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:57.188908 28705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:57.195430 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:57.195451 28718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:57.195744 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:57.195971 28705 server_base.cc:1034] running on GCE node
I20260810 23:00:57.196177 28705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:57.196452 28705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:57.197667 28705 hybrid_clock.cc:648] HybridClock initialized: now 1786402857197643 us; error 36 us; skew 500 ppm
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:57.201279 28705 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:00:57.201974 28705 webserver.cc:460] Webserver started at http://127.6.47.130:36707/ using document root <none> and password file <none>
I20260810 23:00:57.202275 28705 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:57.202354 28705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:57.202582 28705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:57.219499 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "89721bbd0a954bc5a02870787726f2e6"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "49461a9cd667bd7d1c2aad33af64a914"
server_key_iv: "42519158242e4260035670ff98d01dc4"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.219990 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "89721bbd0a954bc5a02870787726f2e6"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "49461a9cd667bd7d1c2aad33af64a914"
server_key_iv: "42519158242e4260035670ff98d01dc4"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.230765 28705 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.000s sys 0.004s
I20260810 23:00:57.248639 28739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.248879 28705 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260810 23:00:57.249027 28705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "89721bbd0a954bc5a02870787726f2e6"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "49461a9cd667bd7d1c2aad33af64a914"
server_key_iv: "42519158242e4260035670ff98d01dc4"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.249150 28705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:57.266214 28705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:57.268636 28705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:57.268822 28705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:57.269100 28705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:57.269518 28705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:57.269573 28705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.269614 28705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:57.269639 28705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.276336 28705 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:43719
I20260810 23:00:57.276717 28947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:43719 every 8 connection(s)
I20260810 23:00:57.277652 28705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:00:57.279407 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 28705
I20260810 23:00:57.279522 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:00:57.279716 6334 external_mini_cluster.cc:1428] Setting key 636c30b6fc4d975736008719854e833e
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:57.288025 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.278473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:38203 (local address 127.6.47.190:42525)
0810 23:00:57.278526 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:57.278529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:57.278728 (+ 199us) server_negotiation.cc:407] Connection header received
0810 23:00:57.279256 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:57.279257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:57.279276 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:57.279325 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:57.280039 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.280761 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.282284 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.282476 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.282550 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.285115 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:57.285117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:57.285118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:57.285125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:57.286594 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.287025 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.287026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.287026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.287107 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.287375 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.287376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.287376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.287493 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:57.287561 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:57.287847 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 23:00:57.287936 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:00:57.288211 28951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.278439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.130:38203)
0810 23:00:57.278610 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:57.278652 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:57.279143 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:57.279396 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:57.279404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:57.279650 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:57.279923 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.279929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.280901 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.280903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.282183 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.282187 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.282591 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.282592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.282734 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.283342 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:57.283352 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:57.284984 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:57.286688 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.286694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.286696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.286942 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.287175 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.287176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.287176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.287297 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.287565 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:57.287579 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:57.287714 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 23:00:57.287826 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 23:00:57.287967 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260810 23:00:57.288810 28949 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:57.288966 28949 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:57.289263 28949 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:57.289710 28329 ts_manager.cc:194] Registered new tserver with Master: 89721bbd0a954bc5a02870787726f2e6 (127.6.47.130:43719)
I20260810 23:00:57.290172 28329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:38203
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:00:57.318837 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:57.424563 28955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:57.424861 28955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:57.424901 28955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:57.426581 28955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:57.426642 28955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:57.426681 28955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:00:57.428673 28955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:57.428901 28955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:57.429211 28955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:57.437892 28965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:57.438336 28963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:57.438421 28967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:57.440254 28955 server_base.cc:1034] running on GCE node
I20260810 23:00:57.440543 28955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:57.440824 28955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:57.448217 28955 hybrid_clock.cc:648] HybridClock initialized: now 1786402857448190 us; error 37 us; skew 500 ppm
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:57.451860 28955 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:00:57.452428 28955 webserver.cc:460] Webserver started at http://127.6.47.131:32947/ using document root <none> and password file <none>
I20260810 23:00:57.452850 28955 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:57.452976 28955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:00:57.453173 28955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:57.468597 28955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "a73eb43faae2423185b9b3762e6989cc"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "f06a0204b6562555f099094384564e64"
server_key_iv: "68e1df9f9bf35dc09a890a157f9fdb85"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.469099 28955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "a73eb43faae2423185b9b3762e6989cc"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "f06a0204b6562555f099094384564e64"
server_key_iv: "68e1df9f9bf35dc09a890a157f9fdb85"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.471736 28955 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 23:00:57.491832 28980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.492060 28955 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260810 23:00:57.492175 28955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "a73eb43faae2423185b9b3762e6989cc"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "f06a0204b6562555f099094384564e64"
server_key_iv: "68e1df9f9bf35dc09a890a157f9fdb85"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:57.492301 28955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:57.533275 28955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:57.535748 28955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:57.535917 28955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:57.536216 28955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:57.536677 28955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:57.536729 28955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.536760 28955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:57.536784 28955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:57.544102 28955 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:34735
I20260810 23:00:57.544239 29167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:34735 every 8 connection(s)
I20260810 23:00:57.544582 28955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:00:57.546693 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 28955
I20260810 23:00:57.546790 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:00:57.546938 6334 external_mini_cluster.cc:1428] Setting key da40282e9c7c0f7fdab32369ae7c644e
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:57.556056 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.545637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:39599 (local address 127.6.47.190:42525)
0810 23:00:57.545678 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:57.545680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:57.545690 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:00:57.546270 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:57.546272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:57.546285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:57.546353 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:57.547129 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.547886 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.549320 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.549609 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.549660 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.552385 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:57.552386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:57.552386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:57.552394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:57.554280 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.554949 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.554951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.554951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.555019 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:57.555317 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:57.555318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:57.555319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:57.555465 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:57.555552 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:57.555870 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 23:00:57.555956 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:00:57.556258 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:57.545315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.131:39599)
0810 23:00:57.545496 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:57.545550 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:57.546131 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:57.546440 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:57.546444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:57.546691 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:57.546972 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.546979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.548022 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.548023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.549175 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:57.549178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:57.549772 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:57.549773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:57.549915 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:57.550550 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:57.550560 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:57.552278 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:57.554417 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.554424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.554427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.554851 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.555101 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:57.555102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:57.555102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:57.555238 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:57.555548 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:57.555554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:57.555711 (+ 157us) client_negotiation.cc:769] Sending connection context
0810 23:00:57.555822 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 23:00:57.555971 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260810 23:00:57.556739 29169 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:57.556898 29169 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:57.557196 29169 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:57.557688 28329 ts_manager.cc:194] Registered new tserver with Master: a73eb43faae2423185b9b3762e6989cc (127.6.47.131:34735)
I20260810 23:00:57.558180 28329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:39599
I20260810 23:00:57.559063 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:00:57.559729 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 28460
I20260810 23:00:57.593765 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 28705
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:57.605971 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 28955
I20260810 23:00:57.624245 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 28273
I20260810 23:00:57.641539 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42525
--webserver_interface=127.6.47.190
--webserver_port=39337
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42525
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:00:57.744239 29173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:57.744437 29173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:57.744457 29173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:57.745537 29173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:00:57.745581 29173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:57.745594 29173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:57.745606 29173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:00:57.745617 29173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:00:57.747166 29173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42525
--ranger_config_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=39337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:57.747438 29173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:57.747663 29173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:57.752735 29191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:57.752782 29186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:57.752869 29173 server_base.cc:1034] running on GCE node
W20260810 23:00:57.752745 29193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:57.753165 29173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:57.753487 29173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:57.754766 29173 hybrid_clock.cc:648] HybridClock initialized: now 1786402857754744 us; error 25 us; skew 500 ppm
Aug 10 23:00:57 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:57.757366 29173 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:00:57.757840 29173 webserver.cc:460] Webserver started at http://127.6.47.190:39337/ using document root <none> and password file <none>
I20260810 23:00:57.758047 29173 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:57.758092 29173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402857, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:58.559821 29173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:58.560732 29204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.560974 29173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:58.561113 29173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "942034e5459b40a9a3016343aca13741"
format_stamp: "Formatted at 2026-08-10 23:00:55 on dist-test-slave-rwrg"
server_key: "45dae84a3b945d7b72cdf602e3620888"
server_key_iv: "7030a820a37de9d6ae2c6339b626aedd"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:58.561499 29173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:58.580440 29173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:58.582113 29173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:58.582291 29173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:58.586745 29173 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42525
I20260810 23:00:58.586793 29292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42525 every 8 connection(s)
I20260810 23:00:58.587203 29173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:00:58.588474 29293 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:00:58.589089 29293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Bootstrap starting.
I20260810 23:00:58.593597 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 29173
I20260810 23:00:58.593899 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:45319
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:42525
--webserver_port=44153
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:00:58.597369 29293 log.cc:824] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Log is configured to *not* fsync() on all Append() calls
I20260810 23:00:58.599109 29293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:00:58.599458 29293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Bootstrap complete.
I20260810 23:00:58.601955 29293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:58.602221 29293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 942034e5459b40a9a3016343aca13741, State: Initialized, Role: FOLLOWER
I20260810 23:00:58.602455 29293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:58.602555 29293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:00:58.602617 29293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:00:58.602666 29293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:00:58.603767 29293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:58.603892 29293 leader_election.cc:302] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 942034e5459b40a9a3016343aca13741; no voters:
I20260810 23:00:58.604074 29293 leader_election.cc:288] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:00:58.604147 29298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:00:58.604352 29293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:00:58.604373 29298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [term 2 LEADER]: Becoming Leader. State: Replica: 942034e5459b40a9a3016343aca13741, State: Running, Role: LEADER
I20260810 23:00:58.604516 29298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } }
I20260810 23:00:58.604821 29298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 942034e5459b40a9a3016343aca13741. Latest consensus state: current_term: 2 leader_uuid: "942034e5459b40a9a3016343aca13741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } } }
I20260810 23:00:58.604941 29298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:58.604825 29301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "942034e5459b40a9a3016343aca13741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "942034e5459b40a9a3016343aca13741" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42525 } } }
I20260810 23:00:58.605293 29301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741 [sys.catalog]: This master's current role is: LEADER
I20260810 23:00:58.605994 29293 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 23:00:58.606290 29314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:00:58.608952 29314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:00:58.609227 29314 catalog_manager.cc:1259] Loaded cluster ID: 9f316fbe4e4f473eb18f12c4344d165d
I20260810 23:00:58.609340 29314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:00:58.611418 29314 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:00:58.611651 29314 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Loaded TSK: 0
I20260810 23:00:58.611912 29314 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:00:58.711719 29296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:58.712169 29296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:58.712291 29296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:58.714152 29296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:58.714279 29296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:58.714372 29296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:00:58.716559 29296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:45319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=44153
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:58.717166 29296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:58.717537 29296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:58.724433 29366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:58.725056 29363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:58.725265 29369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:58.726584 29296 server_base.cc:1034] running on GCE node
I20260810 23:00:58.726878 29296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:58.727164 29296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:58.728257 29296 hybrid_clock.cc:648] HybridClock initialized: now 1786402858728211 us; error 60 us; skew 500 ppm
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:58.731637 29296 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:00:58.734535 29296 webserver.cc:460] Webserver started at http://127.6.47.129:44153/ using document root <none> and password file <none>
I20260810 23:00:58.734920 29296 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:58.735085 29296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:58.751571 29296 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 23:00:58.752986 29380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.753149 29296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:58.753223 29296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0"
format_stamp: "Formatted at 2026-08-10 23:00:56 on dist-test-slave-rwrg"
server_key: "4b3a5b5b1a60d9539990741eeffe15b3"
server_key_iv: "27ca770d90ee55bdedbcc244f74ebb5b"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:58.754932 29296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:58.776738 29296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:58.778829 29296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:58.778999 29296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:58.779230 29296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:58.779573 29296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:58.779616 29296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.779640 29296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:58.779686 29296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.785867 29296 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:45319
I20260810 23:00:58.786213 29296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:00:58.786619 29565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:45319 every 8 connection(s)
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:58.795822 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 29296
I20260810 23:00:58.796155 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:43719
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:42525
--webserver_port=36707
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:00:58.801249 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:58.787714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:55339 (local address 127.6.47.190:42525)
0810 23:00:58.788329 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:58.788339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:58.788385 (+ 46us) server_negotiation.cc:407] Connection header received
0810 23:00:58.788568 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:58.788575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:58.788637 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:58.788778 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:58.789570 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.791022 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.792472 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.793141 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.793264 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:58.796940 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:58.796945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:58.796946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:58.796959 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:58.798950 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:58.799522 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:58.799527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:58.799527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:58.799656 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:58.800016 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:58.800017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:58.800017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:58.800365 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:58.800466 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:58.800886 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 23:00:58.801022 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":78,"threads_started":1}
I20260810 23:00:58.801261 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:58.787293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.129:55339)
0810 23:00:58.787484 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:58.787525 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:58.788165 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:58.788880 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:58.788885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:58.789126 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:58.789436 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:58.789442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.791154 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.791155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:58.792376 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:58.792380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.793339 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.793340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:58.793450 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:58.794067 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:58.794077 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:58.796807 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:58.799066 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:58.799075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:58.799078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:58.799411 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:58.799739 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:58.799740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:58.799741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:58.799930 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:58.800455 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:58.800463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:58.800659 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 23:00:58.800792 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 23:00:58.800970 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260810 23:00:58.802201 29569 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:58.802316 29569 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.802551 29569 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:58.803555 29569 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:42525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:00:58.803628 29569 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.803716 29569 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:58.804148 29569 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.804225 29569 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:58.804539 29569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:00:58.905329 29573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:58.905592 29573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:58.905638 29573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:58.907259 29573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:58.907325 29573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:58.907368 29573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:00:58.909583 29573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:43719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=36707
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:58.909873 29573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:58.910120 29573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:58.915318 29585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:58.915328 29586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:58.915622 29589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:58.915861 29573 server_base.cc:1034] running on GCE node
I20260810 23:00:58.916086 29573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:58.916356 29573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:58.917543 29573 hybrid_clock.cc:648] HybridClock initialized: now 1786402858917541 us; error 42 us; skew 500 ppm
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:58.920884 29573 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:00:58.921399 29573 webserver.cc:460] Webserver started at http://127.6.47.130:36707/ using document root <none> and password file <none>
I20260810 23:00:58.921687 29573 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:58.921775 29573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:58.938256 29573 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:58.941907 29606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.942142 29573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260810 23:00:58.942230 29573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "89721bbd0a954bc5a02870787726f2e6"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "49461a9cd667bd7d1c2aad33af64a914"
server_key_iv: "42519158242e4260035670ff98d01dc4"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:58.942687 29573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:58.974260 29573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:58.976421 29573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:58.976612 29573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:58.976938 29573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:58.977350 29573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:58.977415 29573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.977456 29573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:58.977483 29573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:58.983615 29573 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:43719
I20260810 23:00:58.983698 29806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:43719 every 8 connection(s)
I20260810 23:00:58.984058 29573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:00:58.986193 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 29573
I20260810 23:00:58.986523 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.47.148:58873/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:34735
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:42525
--webserver_port=32947
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 23:00:58 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:58.995100 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:58.984943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:59817 (local address 127.6.47.190:42525)
0810 23:00:58.985024 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:58.985026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:58.985270 (+ 244us) server_negotiation.cc:407] Connection header received
0810 23:00:58.985929 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:58.985930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:58.985942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:58.985984 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:58.986753 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.987544 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.989832 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.990054 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.990114 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:58.992268 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:58.992269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:58.992270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:58.992281 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:58.993713 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:58.994135 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:58.994137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:58.994137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:58.994204 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:58.994461 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:58.994462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:58.994463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:58.994576 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:58.994646 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:58.994923 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 23:00:58.995011 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 23:00:58.995467 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:58.984852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.130:59817)
0810 23:00:58.985115 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:58.985163 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:58.985815 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:58.986063 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:58.986067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:58.986320 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:58.986628 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:58.986635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.987686 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.987687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:58.989724 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:58.989727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:58.990175 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:58.990176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:58.990319 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:58.990910 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:58.990918 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:58.992168 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:58.993818 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:58.993825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:58.993827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:58.994047 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:58.994277 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:58.994277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:58.994278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:58.994387 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:58.994649 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:58.994656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:58.994794 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 23:00:58.994903 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:00:58.995040 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":83,"threads_started":1}
I20260810 23:00:58.996057 29809 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:58.996187 29809 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.996381 29809 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:58.997088 29809 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:42525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:00:58.997162 29809 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.997255 29809 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:58.997560 29809 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:58.997640 29809 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:58.998086 29809 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:00:59.113451 29815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:00:59.113737 29815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:00:59.113776 29815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:00:59.115450 29815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:00:59.115512 29815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:00:59.115551 29815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:00:59.117717 29815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.47.148:58873/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:34735
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=32947
--tserver_master_addrs=127.6.47.190:42525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:00:59.117969 29815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:00:59.118228 29815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:00:59.124553 29833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:59.124588 29831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:00:59.124558 29829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:00:59.125243 29815 server_base.cc:1034] running on GCE node
I20260810 23:00:59.125468 29815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:00:59.125759 29815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:00:59.126942 29815 hybrid_clock.cc:648] HybridClock initialized: now 1786402859126912 us; error 44 us; skew 500 ppm
Aug 10 23:00:59 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:59.130139 29815 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:00:59.130698 29815 webserver.cc:460] Webserver started at http://127.6.47.131:32947/ using document root <none> and password file <none>
I20260810 23:00:59.130990 29815 fs_manager.cc:359] Metadata directory not provided
I20260810 23:00:59.131074 29815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 23:00:59 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:00:59.150096 29815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:59.151161 29843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:59.151361 29815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:00:59.151480 29815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "a73eb43faae2423185b9b3762e6989cc"
format_stamp: "Formatted at 2026-08-10 23:00:57 on dist-test-slave-rwrg"
server_key: "f06a0204b6562555f099094384564e64"
server_key_iv: "68e1df9f9bf35dc09a890a157f9fdb85"
server_key_version: "kuduclusterkey@0"
I20260810 23:00:59.151968 29815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:00:59.169754 29815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:00:59.172154 29815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:00:59.172341 29815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:00:59.172634 29815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:00:59.172978 29815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:00:59.173034 29815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:59.173077 29815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:00:59.173110 29815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:00:59.179690 29815 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:34735
I20260810 23:00:59.179812 30033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:34735 every 8 connection(s)
I20260810 23:00:59.180136 29815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:00:59.185323 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 29815
Aug 10 23:00:59 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:00:59.190997 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:59.181062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:40851 (local address 127.6.47.190:42525)
0810 23:00:59.181147 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:59.181151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:59.181426 (+ 275us) server_negotiation.cc:407] Connection header received
0810 23:00:59.182056 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:59.182057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:59.182067 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:59.182134 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:59.182793 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.183310 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.184315 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.184573 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.184657 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:59.187338 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:59.187339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:59.187339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:59.187346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:59.189131 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:59.189744 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:59.189745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:59.189746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:59.189813 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:59.190182 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:59.190183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:59.190184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:59.190320 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:59.190369 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:59.190777 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 23:00:59.190867 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:00:59.191223 30042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:59.181055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:42525 (local address 127.6.47.131:40851)
0810 23:00:59.181259 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 23:00:59.181331 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 23:00:59.181957 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:00:59.182216 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:00:59.182220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:00:59.182442 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:00:59.182711 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:59.182716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.183410 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.183411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:59.184212 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:00:59.184215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.184687 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.184689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:00:59.184806 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:59.185368 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:00:59.185379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:00:59.187243 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:00:59.189274 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:59.189282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:59.189284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:59.189611 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:59.189918 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:00:59.189919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:00:59.189919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:00:59.190070 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:00:59.190438 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:00:59.190446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:00:59.190607 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 23:00:59.190728 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 23:00:59.190969 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260810 23:00:59.192873 30039 heartbeater.cc:347] Connected to a master server at 127.6.47.190:42525
I20260810 23:00:59.192982 30039 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:59.193203 30039 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:59.193917 30039 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:42525 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 23:00:59.193986 30039 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:59.194042 30039 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:59.194447 30039 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:59.194533 30039 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
W20260810 23:00:59.195878 30039 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:00:59.201555 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:00:59.185764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.6.47.190:42525)
0810 23:00:59.185890 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 23:00:59.185892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:00:59.186086 (+ 194us) server_negotiation.cc:407] Connection header received
0810 23:00:59.186110 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:00:59.186110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:00:59.186122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:00:59.186184 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:00:59.186969 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.187542 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.189320 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:00:59.189542 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:00:59.189641 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:00:59.191173 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:00:59.191174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:00:59.191174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:00:59.191181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:00:59.194290 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:59.196007 (+ 1717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:59.196008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:59.196008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:59.196074 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:00:59.200846 (+ 4772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:00:59.200847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:00:59.200848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:00:59.201018 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:00:59.201220 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 23:00:59.201341 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 23:00:59.201452 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
Aug 10 23:00:59 dist-test-slave-rwrg krb5kdc[27491](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:00:59.804718 29569 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:59.804878 29569 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:59.805291 29243 ts_manager.cc:194] Registered new tserver with Master: 1b6e99724c944aaaaedc1d0c9ccea6b0 (127.6.47.129:45319)
I20260810 23:00:59.806257 29243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:55339
I20260810 23:00:59.998287 29809 heartbeater.cc:464] Registering TS with master...
I20260810 23:00:59.998456 29809 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:00:59.998765 29243 ts_manager.cc:194] Registered new tserver with Master: 89721bbd0a954bc5a02870787726f2e6 (127.6.47.130:43719)
I20260810 23:00:59.999302 29243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:59817
Aug 10 23:01:00 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:01:00.196084 30039 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:00.196242 30039 heartbeater.cc:511] Master 127.6.47.190:42525 requested a full tablet report, sending...
I20260810 23:01:00.196571 29243 ts_manager.cc:194] Registered new tserver with Master: a73eb43faae2423185b9b3762e6989cc (127.6.47.131:34735)
I20260810 23:01:00.197044 29243 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:40851
I20260810 23:01:00.206130 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:01:00.222949 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.211980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.6.47.190:42525)
0810 23:01:00.212153 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.212155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.212228 (+ 73us) server_negotiation.cc:407] Connection header received
0810 23:01:00.212300 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.212301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.212314 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.212390 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:00.213136 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.213909 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.214968 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.215292 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.215370 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.219574 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:00.219576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:00.219576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:00.219585 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:00.221473 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:00.221902 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:00.221904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:00.221904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:00.221976 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:00.222204 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:00.222206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:00.222206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:00.222361 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:00.222471 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.222621 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.222713 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260810 23:01:00.225286 29243 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:01:00.227747 29243 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:01:00.237260 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.231025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.6.47.131:34735)
0810 23:01:00.231375 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.231381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.232752 (+ 1371us) server_negotiation.cc:407] Connection header received
0810 23:01:00.232837 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.232839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.232858 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.232939 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:00.233208 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.234377 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.235936 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.236746 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.236814 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.236869 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.237132 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.237176 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":70,"threads_started":1}
I20260810 23:01:00.237756 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.230845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:34735 (local address 127.0.0.1:60738)
0810 23:01:00.232625 (+ 1780us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.232670 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:00.232751 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:00.233046 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:00.233050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:00.233052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:00.233123 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.233124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.234480 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.234481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.235832 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.235835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.236933 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.236934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.237015 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.237029 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:01:00.237600 (+ 571us) client_negotiation.cc:240] Negotiation successful
0810 23:01:00.237676 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1746,"mutex_wait_us":28,"thread_start_us":97,"threads_started":1}
I20260810 23:01:00.237777 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.231190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58696 (local address 127.6.47.130:43719)
0810 23:01:00.231463 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.231467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.232937 (+ 1470us) server_negotiation.cc:407] Connection header received
0810 23:01:00.233053 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.233055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.233084 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.233140 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:00.233395 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.234420 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.236146 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.237277 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.237325 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.237377 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.237657 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.237699 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":56,"threads_started":1}
I20260810 23:01:00.238034 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.230853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43719 (local address 127.0.0.1:58696)
0810 23:01:00.232871 (+ 2018us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.232880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:00.232965 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:00.233253 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:00.233254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:00.233257 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:00.233312 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.233313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.234778 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.234778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.236025 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.236029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.237446 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.237446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.237525 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.237529 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:00.237916 (+ 387us) client_negotiation.cc:240] Negotiation successful
0810 23:01:00.237963 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2002,"thread_start_us":107,"threads_started":1}
I20260810 23:01:00.238744 29900 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 (DEFAULT_TABLE table=test-table [id=d3409cade3b74ef982588bc691029382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:00.238780 29690 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 (DEFAULT_TABLE table=test-table [id=d3409cade3b74ef982588bc691029382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:00.239122 29900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac6afd965f84fe7b697e267fcaf3cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:00.239143 29690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac6afd965f84fe7b697e267fcaf3cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:00.242499 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.231222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.6.47.129:45319)
0810 23:01:00.231608 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.231612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.238195 (+ 6583us) server_negotiation.cc:407] Connection header received
0810 23:01:00.238323 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.238326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.238342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.238400 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:00.238616 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.239917 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.241183 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.241945 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.241997 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.242051 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.242313 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.242420 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":60,"threads_started":1}
I20260810 23:01:00.242729 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.231100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45319 (local address 127.0.0.1:45096)
0810 23:01:00.238119 (+ 7019us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.238130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:00.238229 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:00.238508 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:00.238509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:00.238510 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:00.238540 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.238541 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.240007 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.240008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.241076 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.241079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.242139 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.242140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.242200 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.242203 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:01:00.242626 (+ 423us) client_negotiation.cc:240] Negotiation successful
0810 23:01:00.242667 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7007,"thread_start_us":66,"threads_started":1}
I20260810 23:01:00.243534 29456 tablet_service.cc:1467] Processing CreateTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 (DEFAULT_TABLE table=test-table [id=d3409cade3b74ef982588bc691029382]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:00.243834 29456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ac6afd965f84fe7b697e267fcaf3cf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:00.246086 30108 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Bootstrap starting.
I20260810 23:01:00.246811 30108 tablet_bootstrap.cc:654] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:00.246806 30110 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Bootstrap starting.
I20260810 23:01:00.247437 30110 tablet_bootstrap.cc:654] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:00.247223 30108 log.cc:824] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:00.247792 30110 log.cc:824] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:00.248385 30108 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: No bootstrap required, opened a new log
I20260810 23:01:00.248469 30110 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: No bootstrap required, opened a new log
I20260810 23:01:00.248461 30108 ts_tablet_manager.cc:1397] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:01:00.248554 30110 ts_tablet_manager.cc:1397] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:01:00.251713 30106 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Bootstrap starting.
I20260810 23:01:00.251643 30108 raft_consensus.cc:348] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.251822 30108 raft_consensus.cc:374] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:00.251752 30110 raft_consensus.cc:348] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.251863 30108 raft_consensus.cc:729] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73eb43faae2423185b9b3762e6989cc, State: Initialized, Role: FOLLOWER
I20260810 23:01:00.251878 30110 raft_consensus.cc:374] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:00.251911 30110 raft_consensus.cc:729] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b6e99724c944aaaaedc1d0c9ccea6b0, State: Initialized, Role: FOLLOWER
I20260810 23:01:00.252017 30108 consensus_queue.cc:260] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.252067 30110 consensus_queue.cc:260] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.252394 30110 ts_tablet_manager.cc:1428] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:01:00.252569 29569 heartbeater.cc:503] Master 127.6.47.190:42525 was elected leader, sending a full tablet report...
I20260810 23:01:00.252394 30108 ts_tablet_manager.cc:1428] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:01:00.252914 30039 heartbeater.cc:503] Master 127.6.47.190:42525 was elected leader, sending a full tablet report...
I20260810 23:01:00.253180 30106 tablet_bootstrap.cc:654] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:00.254726 30106 log.cc:824] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:00.255534 30106 tablet_bootstrap.cc:492] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: No bootstrap required, opened a new log
I20260810 23:01:00.255622 30106 ts_tablet_manager.cc:1397] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 23:01:00.259037 30106 raft_consensus.cc:348] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.259233 30106 raft_consensus.cc:374] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:00.259284 30106 raft_consensus.cc:729] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89721bbd0a954bc5a02870787726f2e6, State: Initialized, Role: FOLLOWER
I20260810 23:01:00.259435 30106 consensus_queue.cc:260] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.259771 29809 heartbeater.cc:503] Master 127.6.47.190:42525 was elected leader, sending a full tablet report...
I20260810 23:01:00.260066 30106 ts_tablet_manager.cc:1428] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
W20260810 23:01:00.288066 29570 tablet.cc:2369] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 23:01:00 dist-test-slave-rwrg krb5kdc[27491](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for HTTP/127.6.47.148@KRBTEST.COM
I20260810 23:01:00.395967 30117 raft_consensus.cc:482] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:00.396194 30117 raft_consensus.cc:504] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.396862 30117 leader_election.cc:288] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a73eb43faae2423185b9b3762e6989cc (127.6.47.131:34735), 89721bbd0a954bc5a02870787726f2e6 (127.6.47.130:43719)
I20260810 23:01:00.402171 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.397460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:48863 (local address 127.6.47.130:43719)
0810 23:01:00.397519 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.397522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.397569 (+ 47us) server_negotiation.cc:407] Connection header received
0810 23:01:00.397684 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.397685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.397698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.397753 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:00.398116 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.398743 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.400797 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.401541 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.401600 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.401610 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.402028 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.402077 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:01:00.402688 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.397114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:43719 (local address 127.6.47.129:48863)
0810 23:01:00.397367 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.397380 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:00.397601 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:00.397835 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:00.397836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:00.397837 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:00.398006 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.398009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.398974 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.398975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.400685 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.400689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.401815 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.401818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.401904 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.401909 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:01:00.402408 (+ 499us) client_negotiation.cc:240] Negotiation successful
0810 23:01:00.402456 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":196,"threads_started":1}
I20260810 23:01:00.403194 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac6afd965f84fe7b697e267fcaf3cf7" candidate_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721bbd0a954bc5a02870787726f2e6" is_pre_election: true
I20260810 23:01:00.403412 29723 raft_consensus.cc:2454] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6e99724c944aaaaedc1d0c9ccea6b0 in term 0.
I20260810 23:01:00.404367 29382 leader_election.cc:302] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6e99724c944aaaaedc1d0c9ccea6b0, 89721bbd0a954bc5a02870787726f2e6; no voters:
I20260810 23:01:00.404565 30117 raft_consensus.cc:2781] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:00.404632 30117 raft_consensus.cc:482] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:00.404662 30117 raft_consensus.cc:3037] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:00.406275 30117 raft_consensus.cc:504] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.406448 30117 leader_election.cc:288] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [CANDIDATE]: Term 1 election: Requested vote from peers a73eb43faae2423185b9b3762e6989cc (127.6.47.131:34735), 89721bbd0a954bc5a02870787726f2e6 (127.6.47.130:43719)
I20260810 23:01:00.406817 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac6afd965f84fe7b697e267fcaf3cf7" candidate_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89721bbd0a954bc5a02870787726f2e6"
I20260810 23:01:00.406908 29723 raft_consensus.cc:3037] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:00.408356 29723 raft_consensus.cc:2454] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6e99724c944aaaaedc1d0c9ccea6b0 in term 1.
I20260810 23:01:00.409612 29382 leader_election.cc:302] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b6e99724c944aaaaedc1d0c9ccea6b0, 89721bbd0a954bc5a02870787726f2e6; no voters:
I20260810 23:01:00.409824 30117 raft_consensus.cc:2781] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:00.410053 30117 raft_consensus.cc:686] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 1b6e99724c944aaaaedc1d0c9ccea6b0, State: Running, Role: LEADER
I20260810 23:01:00.410168 30117 consensus_queue.cc:237] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } }
I20260810 23:01:00.411305 29243 catalog_manager.cc:5495] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b6e99724c944aaaaedc1d0c9ccea6b0 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45319 } health_report { overall_health: HEALTHY } } }
I20260810 23:01:00.414413 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.397895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50929 (local address 127.6.47.131:34735)
0810 23:01:00.397960 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.397962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.403121 (+ 5159us) server_negotiation.cc:407] Connection header received
0810 23:01:00.403224 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.403225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.403240 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.403272 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:00.403458 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.404131 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.406004 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.406638 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.406685 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.406695 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.414249 (+ 7554us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.414307 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:01:00.417235 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.397648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:34735 (local address 127.6.47.129:50929)
0810 23:01:00.403064 (+ 5416us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.403076 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:00.403149 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:00.403335 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:00.403335 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:00.403337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:00.403367 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.403368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.404559 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.404560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.405871 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:00.405875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.413979 (+ 8104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.413981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:00.414059 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.414063 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:00.414688 (+ 625us) client_negotiation.cc:240] Negotiation successful
0810 23:01:00.417047 (+ 2359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5401,"thread_start_us":143,"threads_started":1}
I20260810 23:01:00.417711 29962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac6afd965f84fe7b697e267fcaf3cf7" candidate_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73eb43faae2423185b9b3762e6989cc" is_pre_election: true
I20260810 23:01:00.417974 29962 raft_consensus.cc:2454] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b6e99724c944aaaaedc1d0c9ccea6b0 in term 0.
I20260810 23:01:00.418412 29961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ac6afd965f84fe7b697e267fcaf3cf7" candidate_uuid: "1b6e99724c944aaaaedc1d0c9ccea6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a73eb43faae2423185b9b3762e6989cc"
I20260810 23:01:00.418522 29961 raft_consensus.cc:3037] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:00.420058 29961 raft_consensus.cc:2454] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b6e99724c944aaaaedc1d0c9ccea6b0 in term 1.
I20260810 23:01:00.426609 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.422658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.6.47.129:45319)
0810 23:01:00.422706 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:00.422708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:00.422716 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:01:00.422756 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:00.422757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:00.422781 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:00.422852 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:00.423063 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.424013 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.425402 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:00.425727 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:00.425793 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:00.426008 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:00.426152 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:00.426232 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:00.426478 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 23:01:00.426517 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 23:01:00.429352 29723 raft_consensus.cc:1261] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 1 FOLLOWER]: Refusing update from remote peer 1b6e99724c944aaaaedc1d0c9ccea6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:00.429468 29961 raft_consensus.cc:1261] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 1 FOLLOWER]: Refusing update from remote peer 1b6e99724c944aaaaedc1d0c9ccea6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:00.430035 30127 consensus_queue.cc:1035] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89721bbd0a954bc5a02870787726f2e6" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 43719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:00.430213 30127 consensus_queue.cc:1035] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 1b6e99724c944aaaaedc1d0c9ccea6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a73eb43faae2423185b9b3762e6989cc" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 34735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 23:01:00.432063 30041 tablet.cc:2369] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:01:00.437944 30133 mvcc.cc:205] Tried to move back new op lower bound from 7317106116315136000 to 7317106116240658432. Current Snapshot: MvccSnapshot[applied={T|T < 7317106116315136000}]
I20260810 23:01:00.438279 30137 mvcc.cc:205] Tried to move back new op lower bound from 7317106116315136000 to 7317106116240658432. Current Snapshot: MvccSnapshot[applied={T|T < 7317106116315136000}]
I20260810 23:01:00.440042 30134 mvcc.cc:205] Tried to move back new op lower bound from 7317106116315136000 to 7317106116240658432. Current Snapshot: MvccSnapshot[applied={T|T < 7317106116315136000}]
I20260810 23:01:00.447197 29240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:00.447314 29240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:00.448338 29240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 942034e5459b40a9a3016343aca13741: Sending DeleteTablet for 3 replicas of tablet 6ac6afd965f84fe7b697e267fcaf3cf7
I20260810 23:01:00.448925 29690 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:00 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:58696
I20260810 23:01:00.449441 29456 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:00 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:45096
I20260810 23:01:00.449787 29900 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ac6afd965f84fe7b697e267fcaf3cf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:00 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:60738
I20260810 23:01:00.450371 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 29296
I20260810 23:01:00.480659 30156 tablet_replica.cc:330] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: stopping tablet replica
I20260810 23:01:00.480855 30156 raft_consensus.cc:2229] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:00.481041 30156 raft_consensus.cc:2258] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:00.481297 30149 tablet_replica.cc:330] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: stopping tablet replica
I20260810 23:01:00.481441 30149 raft_consensus.cc:2229] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:00.481607 30149 raft_consensus.cc:2258] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:00.481782 30156 ts_tablet_manager.cc:1905] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:00.483157 30149 ts_tablet_manager.cc:1905] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:00.484193 30156 ts_tablet_manager.cc:1918] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:00.484280 30156 log.cc:1196] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/6ac6afd965f84fe7b697e267fcaf3cf7
I20260810 23:01:00.484635 30156 ts_tablet_manager.cc:1939] T 6ac6afd965f84fe7b697e267fcaf3cf7 P a73eb43faae2423185b9b3762e6989cc: Deleting consensus metadata
I20260810 23:01:00.485139 29211 catalog_manager.cc:4841] TS a73eb43faae2423185b9b3762e6989cc (127.6.47.131:34735): tablet 6ac6afd965f84fe7b697e267fcaf3cf7 (table test-table [id=d3409cade3b74ef982588bc691029382]) successfully deleted
I20260810 23:01:00.485167 30149 ts_tablet_manager.cc:1918] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:00.485334 30149 log.cc:1196] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/6ac6afd965f84fe7b697e267fcaf3cf7
I20260810 23:01:00.485639 30149 ts_tablet_manager.cc:1939] T 6ac6afd965f84fe7b697e267fcaf3cf7 P 89721bbd0a954bc5a02870787726f2e6: Deleting consensus metadata
I20260810 23:01:00.486114 29209 catalog_manager.cc:4841] TS 89721bbd0a954bc5a02870787726f2e6 (127.6.47.130:43719): tablet 6ac6afd965f84fe7b697e267fcaf3cf7 (table test-table [id=d3409cade3b74ef982588bc691029382]) successfully deleted
W20260810 23:01:00.493726 29209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:45319 (error 108)
I20260810 23:01:00.494580 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 29573
I20260810 23:01:00.494724 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:00.494581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45319 (local address 127.0.0.1:45112)
0810 23:01:00.494638 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:00.494691 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 23:01:00.502403 29209 catalog_manager.cc:4584] TS 1b6e99724c944aaaaedc1d0c9ccea6b0 (127.6.47.129:45319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ac6afd965f84fe7b697e267fcaf3cf7: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45319: connect: Connection refused (error 111)
I20260810 23:01:00.517035 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 29815
I20260810 23:01:00.529807 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 29173
I20260810 23:01:00.555873 6334 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 23:01:00.901166 6334 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 23:01:00.901263 6334 mini_ranger.cc:66] Stopping Ranger...
I20260810 23:01:01.246718 6334 mini_ranger.cc:68] Stopped Ranger
2026-08-10 23:01:01.247 UTC [27555] LOG: received smart shutdown request
2026-08-10 23:01:01.251 UTC [27555] LOG: background worker "logical replication launcher" (PID 27568) exited with exit code 1
2026-08-10 23:01:01.251 UTC [27561] LOG: shutting down
2026-08-10 23:01:01.417 UTC [27555] LOG: database system is shut down
2026-08-10T23:01:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50542 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:01:01 dist-test-slave-rwrg krb5kdc[30174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:01:01 dist-test-slave-rwrg krb5kdc[30174](info): set up 2 sockets
Aug 10 23:01:01 dist-test-slave-rwrg krb5kdc[30174](info): commencing operation
krb5kdc: starting...
W20260810 23:01:02.810545 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:01:02 dist-test-slave-rwrg krb5kdc[30174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:01:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:01:02.933594 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46753
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:03.016014 30194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:03.016239 30194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:03.016263 30194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:03.017624 30194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:03.017668 30194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:03.017681 30194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:03.017694 30194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:03.017705 30194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:03.019218 30194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:03.019474 30194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:03.019722 30194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:03.024864 30206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:03.024825 30199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:03.024973 30194 server_base.cc:1034] running on GCE node
W20260810 23:01:03.024847 30213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:03.025394 30194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:03.025709 30194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:03.026887 30194 hybrid_clock.cc:648] HybridClock initialized: now 1786402863026872 us; error 29 us; skew 500 ppm
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:03.029412 30194 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:03.029826 30194 webserver.cc:460] Webserver started at http://127.6.47.190:34387/ using document root <none> and password file <none>
I20260810 23:01:03.030026 30194 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:03.030066 30194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:03.030154 30194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:03.031168 30194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "e0edd9e817124fe097e552f51214c61b"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "9cabbc2aca8092cc8cd9ca01682918d3"
server_key_iv: "825420ef354c36c5ec2c78623d26c90f"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.031483 30194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "e0edd9e817124fe097e552f51214c61b"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "9cabbc2aca8092cc8cd9ca01682918d3"
server_key_iv: "825420ef354c36c5ec2c78623d26c90f"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.032814 30194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:01:03.033543 30224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:03.033746 30194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:03.033838 30194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "e0edd9e817124fe097e552f51214c61b"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "9cabbc2aca8092cc8cd9ca01682918d3"
server_key_iv: "825420ef354c36c5ec2c78623d26c90f"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.033897 30194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:03.044108 30194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:03.046411 30194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:03.046525 30194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:03.050556 30194 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:46753
I20260810 23:01:03.050618 30325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:46753 every 8 connection(s)
I20260810 23:01:03.050972 30194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:03.051909 30326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:03.053565 30326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.058475 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 30194
I20260810 23:01:03.058619 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:01:03.058804 6334 external_mini_cluster.cc:1428] Setting key b6819600e0aab8e6a6f3e02b420332f9
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:03.063410 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.054364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.6.47.190:46753)
0810 23:01:03.054477 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.054482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.054519 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:01:03.055050 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.055051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.055077 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.055179 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.055734 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.056617 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.057531 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.057849 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.057980 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.060147 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.060156 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.060157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.060165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.061837 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.062294 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.062296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.062296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.062375 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.062657 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.062658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.062658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.062852 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.062914 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.063150 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.063258 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":41,"threads_started":1}
I20260810 23:01:03.063578 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.054212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:46753 (local address 127.0.0.1:38722)
0810 23:01:03.054372 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.054422 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.054883 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.055186 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.055189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.055476 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.055627 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.055631 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.056741 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.056742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.057426 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.057427 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.057955 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.057956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.058021 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.058514 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.058525 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.060051 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.061927 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.061934 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.061936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.062207 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.062458 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.062459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.062459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.062575 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.062923 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.062931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.063011 (+ 80us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.063112 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.063381 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":85,"thread_start_us":68,"threads_started":1}
I20260810 23:01:03.064376 30326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.064769 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.064616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:42984)
0810 23:01:03.064678 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.064733 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:01:03.064925 30229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
I20260810 23:01:03.065428 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.065310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:42998)
0810 23:01:03.065363 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.065396 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:01:03.065590 30326 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:37419: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.6.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
I20260810 23:01:03.084772 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:37419
--webserver_interface=127.6.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:03.114859 30326 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } attempt: 1
I20260810 23:01:03.115438 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.115266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43002)
0810 23:01:03.115350 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.115398 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 23:01:03.115820 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.115720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43018)
0810 23:01:03.115769 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.115798 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:01:03.115942 30326 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:37419: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
W20260810 23:01:03.167649 30338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:03.167865 30338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:03.167886 30338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:03.169131 30338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:03.169188 30338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:03.169202 30338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:03.169215 30338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:03.169227 30338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:03.170782 30338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:03.171052 30338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:03.171301 30338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:03.176249 30350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:03.176272 30346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:03.176306 30347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:03.176350 30338 server_base.cc:1034] running on GCE node
I20260810 23:01:03.176810 30338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:03.177059 30338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:03.178241 30338 hybrid_clock.cc:648] HybridClock initialized: now 1786402863178229 us; error 44 us; skew 500 ppm
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:03.180763 30338 init.cc:377] Logged in from keytab as kudu/127.6.47.189@KRBTEST.COM (short username kudu)
I20260810 23:01:03.181195 30338 webserver.cc:460] Webserver started at http://127.6.47.189:35867/ using document root <none> and password file <none>
I20260810 23:01:03.181422 30338 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:03.181490 30338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:03.181653 30338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:03.182749 30338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data/instance:
uuid: "71905b6b651b46fe931a5a2f38a8f91d"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "55c80e30eecee86ed674ac22b59553cd"
server_key_iv: "64a31192f52dd7d15cd82e5eb5ffed2c"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.183128 30338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal/instance:
uuid: "71905b6b651b46fe931a5a2f38a8f91d"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "55c80e30eecee86ed674ac22b59553cd"
server_key_iv: "64a31192f52dd7d15cd82e5eb5ffed2c"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.184147 30326 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } attempt: 2
I20260810 23:01:03.184602 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.184437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43034)
0810 23:01:03.184536 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.184570 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:01:03.184645 30338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:01:03.185006 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.184900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43036)
0810 23:01:03.184954 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.184980 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:01:03.185164 30326 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:37419: Network error: Client connection negotiation failed: client connection to 127.6.47.189:37419: connect: Connection refused (error 111)
I20260810 23:01:03.185622 30362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:03.185812 30338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:03.185912 30338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "71905b6b651b46fe931a5a2f38a8f91d"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "55c80e30eecee86ed674ac22b59553cd"
server_key_iv: "64a31192f52dd7d15cd82e5eb5ffed2c"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.185989 30338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:03.207739 30338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:03.210276 30338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:03.210450 30338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:03.214512 30338 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.189:37419
I20260810 23:01:03.214581 30445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.189:37419 every 8 connection(s)
I20260810 23:01:03.214943 30338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
I20260810 23:01:03.215889 30447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:03.217839 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.219790 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 30338
I20260810 23:01:03.219954 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-1/wal/instance
I20260810 23:01:03.220216 6334 external_mini_cluster.cc:1428] Setting key 7fe2241ac4e4c244fc5e86089fbf79e7
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:03.228607 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.218668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.6.47.190:46753)
0810 23:01:03.218752 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.218754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.218966 (+ 212us) server_negotiation.cc:407] Connection header received
0810 23:01:03.219437 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.219438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.219449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.219507 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.220550 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.221131 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.222491 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.222826 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.222920 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.225411 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.225413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.225413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.225420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.226857 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.227557 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.227558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.227559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.227619 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.227931 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.227932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.227932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.228032 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.228083 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.228421 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.228490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 23:01:03.228863 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.218600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:46753 (local address 127.0.0.1:38732)
0810 23:01:03.218850 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.218885 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.219348 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.219658 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.219663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.220094 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.220410 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.220418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.221235 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.221237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.222397 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.222400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.222931 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.222932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.223039 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.223556 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.223565 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.225298 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.226951 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.226959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.226961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.227316 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.227706 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.227707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.227707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.227854 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.228110 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.228118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.228278 (+ 160us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.228403 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.228615 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.6.47.188@KRBTEST.COM; defaulting to no policy
I20260810 23:01:03.229522 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.188@KRBTEST.COM" created.
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:01:03.237000 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.229839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.6.47.189:37419)
0810 23:01:03.229999 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.230004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.230028 (+ 24us) server_negotiation.cc:407] Connection header received
0810 23:01:03.230063 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.230080 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.230091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.230172 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.230437 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.231193 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.231870 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.232171 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.232251 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.234094 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.234103 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.234103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.234110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.235566 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.235994 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.235996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.235996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.236073 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.236333 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.236334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.236334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.236457 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.236610 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.236782 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.236916 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260810 23:01:03.237002 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.229791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43050)
0810 23:01:03.229845 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.229855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.229899 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.230182 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.230183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.230321 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.230361 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.230363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.231303 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.231304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.231788 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.231790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.232260 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.232261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.232334 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.232747 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.232752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.233996 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.235655 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.235657 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.235657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.235907 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.236160 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.236161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.236161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.236260 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.236617 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.236618 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.236667 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.236752 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.236860 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 23:01:03.237720 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.238071 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.237939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:54994)
0810 23:01:03.237989 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.238043 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:01:03.238231 30365 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
I20260810 23:01:03.238675 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.238562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:54998)
0810 23:01:03.238617 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.238648 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:01:03.238792 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:46821: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
I20260810 23:01:03.247731 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:46821
--webserver_interface=127.6.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:03.276329 30326 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } attempt: 3
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:01:03.283558 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.276820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.6.47.189:37419)
0810 23:01:03.276892 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.276894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.276908 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:01:03.276939 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.276941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.276950 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.277024 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.277381 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.277889 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.278607 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.278823 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.278906 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.281004 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.281006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.281007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.281015 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.282396 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.282740 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.282741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.282742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.282801 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.283018 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.283019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.283019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.283129 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.283211 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.283364 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.283458 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:01:03.283558 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.276725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43052)
0810 23:01:03.276795 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.276807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.276881 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.277030 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.277031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.277223 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.277288 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.277290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.277986 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.277988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.278521 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.278522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.278902 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.278903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.278988 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.279486 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.279490 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.280915 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.282485 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.282486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.282487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.282663 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.282885 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.282886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.282886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.282951 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.283203 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.283204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.283252 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.283338 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.283457 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 23:01:03.284032 30326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.284394 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.284289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55002)
0810 23:01:03.284341 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.284368 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:01:03.284857 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.284771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55008)
0810 23:01:03.284816 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.284841 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:01:03.284965 30326 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:46821: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
I20260810 23:01:03.288019 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } attempt: 1
I20260810 23:01:03.288420 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.288302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55018)
0810 23:01:03.288358 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.288394 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:01:03.288766 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.288671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55024)
0810 23:01:03.288721 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.288748 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:01:03.288861 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:46821: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
I20260810 23:01:03.316102 30326 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } attempt: 1
I20260810 23:01:03.316577 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.316424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55036)
0810 23:01:03.316484 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.316540 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:01:03.316906 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.316811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55052)
0810 23:01:03.316854 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.316877 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 23:01:03.316994 30326 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:46821: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
W20260810 23:01:03.332888 30458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:03.333127 30458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:03.333165 30458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:03.334285 30458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:03.334352 30458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:03.334375 30458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:03.334403 30458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:03.334434 30458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:03.335973 30458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:46753,127.6.47.189:37419,127.6.47.188:46821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:46821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:03.336263 30458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:03.336552 30458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:03.341492 30468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:03.341512 30469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:03.341765 30458 server_base.cc:1034] running on GCE node
W20260810 23:01:03.341499 30472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:03.342118 30458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:03.342422 30458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:03.343592 30458 hybrid_clock.cc:648] HybridClock initialized: now 1786402863343580 us; error 31 us; skew 500 ppm
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:03.346155 30458 init.cc:377] Logged in from keytab as kudu/127.6.47.188@KRBTEST.COM (short username kudu)
I20260810 23:01:03.346753 30458 webserver.cc:460] Webserver started at http://127.6.47.188:39309/ using document root <none> and password file <none>
I20260810 23:01:03.347039 30458 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:03.347169 30458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:03.347321 30458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:03.348397 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data/instance:
uuid: "65c0cfffcfa643c1bc7dd3448710f0a8"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "6f7e008fb1e3bc8316d00409c3a5ff34"
server_key_iv: "78cb447bdc740fb2b4d588b510fa1759"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.348857 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal/instance:
uuid: "65c0cfffcfa643c1bc7dd3448710f0a8"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "6f7e008fb1e3bc8316d00409c3a5ff34"
server_key_iv: "78cb447bdc740fb2b4d588b510fa1759"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.350252 30458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:03.350997 30485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:03.351203 30458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:03.351300 30458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "65c0cfffcfa643c1bc7dd3448710f0a8"
format_stamp: "Formatted at 2026-08-10 23:01:03 on dist-test-slave-rwrg"
server_key: "6f7e008fb1e3bc8316d00409c3a5ff34"
server_key_iv: "78cb447bdc740fb2b4d588b510fa1759"
server_key_version: "encryptionkey@0"
I20260810 23:01:03.351378 30458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:03.357012 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } attempt: 2
I20260810 23:01:03.357463 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.357336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55064)
0810 23:01:03.357398 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.357437 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:01:03.357827 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.357733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55072)
0810 23:01:03.357778 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.357804 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:01:03.357913 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:46821: Network error: Client connection negotiation failed: client connection to 127.6.47.188:46821: connect: Connection refused (error 111)
I20260810 23:01:03.366020 30458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:03.367974 30458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:03.368142 30458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:03.372378 30458 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.188:46821
I20260810 23:01:03.372416 30572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.188:46821 every 8 connection(s)
I20260810 23:01:03.372840 30458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
I20260810 23:01:03.373734 30577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:03.375219 30577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:03.382578 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 30458
I20260810 23:01:03.382727 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786402701710467-6334-0/minicluster-data/master-2/wal/instance
I20260810 23:01:03.382943 6334 external_mini_cluster.cc:1428] Setting key 45542aa59bc996a93cfa2e23e98fd51e
I20260810 23:01:03.384018 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.375909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.6.47.190:46753)
0810 23:01:03.375967 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.375969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.376144 (+ 175us) server_negotiation.cc:407] Connection header received
0810 23:01:03.376627 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.376629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.376637 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.376701 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.377264 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.377711 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.378686 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.378873 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.378925 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.381198 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.381199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.381200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.381206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.382412 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.382915 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.382916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.382916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.382970 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.383313 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.383315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.383315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.383450 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.383534 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.383845 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.383930 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:01:03.384219 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.375850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:46753 (local address 127.0.0.1:38736)
0810 23:01:03.376010 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.376051 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.376483 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.376757 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.376760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.376996 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.377191 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.377195 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.377808 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.377808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.378610 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.378612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.378945 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.378946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.379040 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.379618 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.379637 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.381098 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.382491 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.382500 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.382502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.382836 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.383045 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.383046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.383046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.383218 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.383527 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.383534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.383693 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.383823 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.383973 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260810 23:01:03.384774 30577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
I20260810 23:01:03.391594 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.385103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.6.47.189:37419)
0810 23:01:03.385149 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.385151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.385160 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:01:03.385186 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.385188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.385197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.385252 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.385509 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.385959 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386595 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.386806 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.388587 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.388588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.388589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.388595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.390491 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.390825 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.390826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.390827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.390888 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.391072 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.391073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.391073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.391214 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.391280 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.391447 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.391509 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:01:03.391721 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.385022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:37419 (local address 127.0.0.1:43054)
0810 23:01:03.385072 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.385083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.385136 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.385254 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.385254 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.385385 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.385430 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.385431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.386039 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.386509 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.386511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.386891 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.386955 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.387272 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.387275 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.388462 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.390581 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.390582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.390583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.390751 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.390961 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.390961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.390962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.391011 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.391286 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.391286 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.391334 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.391414 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.391526 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:01:03.392083 30577 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:01:03.392109 30326 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } attempt: 2
I20260810 23:01:03.393047 30588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.383576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.6.47.188:46821)
0810 23:01:03.383743 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.383747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.383764 (+ 17us) server_negotiation.cc:407] Connection header received
0810 23:01:03.383781 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.383782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.383792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.383871 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.384645 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.385454 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386174 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.386453 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.386533 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.389214 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.389223 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.389223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.389231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.391185 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.391615 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.391618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.391619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.391683 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.391887 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.391888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.391889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.392082 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.392815 (+ 733us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.392867 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.392977 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:03.401122 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.392351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55106)
0810 23:01:03.392403 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.392413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.392447 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.392824 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.392825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.393008 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.393036 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.393037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.393722 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.393723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.394491 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.394493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.394961 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.394962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.395015 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.395699 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.395703 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.397008 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.398798 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.398799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.398799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.399076 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.399537 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.399538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.399539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.399600 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.400735 (+ 1135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.400737 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.400801 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.400891 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.400989 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:01:03.401257 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.392580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55106 (local address 127.6.47.188:46821)
0810 23:01:03.392689 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.392691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.392696 (+ 5us) server_negotiation.cc:407] Connection header received
0810 23:01:03.392711 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.392712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.392724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.392790 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.393120 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.393626 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.394566 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.394877 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.394946 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.397126 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.397127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.397127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.397132 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.398711 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.399393 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.399394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.399395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.399434 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.399911 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.399912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.399913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.400070 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.400138 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.400929 (+ 791us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.401107 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":93,"thread_start_us":74,"threads_started":1}
I20260810 23:01:03.401335 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.392363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55094)
0810 23:01:03.392571 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.392583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.392638 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.392819 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.392820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.392980 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.393024 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.393026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.394428 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.394429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.395185 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.395188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.395626 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.395627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.395666 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.396056 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.396061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.397806 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.400211 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.400212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.400212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.400404 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.400645 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.400645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.400646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.400703 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.401033 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.401034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.401079 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.401142 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.401227 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260810 23:01:03.401727 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.392389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55094 (local address 127.6.47.188:46821)
0810 23:01:03.392592 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.392593 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.392624 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:01:03.392711 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.392712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.392724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.393097 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.393354 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.394345 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.395263 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.395547 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.395614 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.397938 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.397939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.397939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.397945 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.400132 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.400478 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.400479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.400480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.400568 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.400784 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.400785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.400786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.400933 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.401498 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.401556 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.401633 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":133,"threads_started":1}
I20260810 23:01:03.402910 30326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: Bootstrap starting.
I20260810 23:01:03.403131 30577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: Bootstrap starting.
I20260810 23:01:03.403519 30326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:03.403680 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.393872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.6.47.190:46753)
0810 23:01:03.393906 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.393908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.393916 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:01:03.393938 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.393939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.393946 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.394011 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.394582 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.395080 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.396566 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.396782 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.396844 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.399202 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.399203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.399204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.399210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.400639 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.401083 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.401084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.401084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.401139 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.402144 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.402146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.402146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.402277 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.403080 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.403395 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.403488 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260810 23:01:03.403883 30577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:03.403769 30326 log.cc:824] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:03.404222 30577 log.cc:824] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:03.404942 30326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: No bootstrap required, opened a new log
I20260810 23:01:03.405001 30577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: No bootstrap required, opened a new log
I20260810 23:01:03.406838 30577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.406839 30326 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.406984 30326 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:03.406985 30577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:03.407028 30326 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0edd9e817124fe097e552f51214c61b, State: Initialized, Role: FOLLOWER
I20260810 23:01:03.407028 30577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c0cfffcfa643c1bc7dd3448710f0a8, State: Initialized, Role: FOLLOWER
I20260810 23:01:03.407160 30326 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.407176 30577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.407460 30326 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:03.407475 30577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:03.407604 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.407685 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.408480 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.408710 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
W20260810 23:01:03.410234 30627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:01:03.410290 30627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:01:03.411036 30194 master_runner.cc:186] Error getting master registration for 127.6.47.189:37419: OK, instance_id { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" instance_seqno: 1786402863213369 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:01:03.413460 30194 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 23:01:03.409437 30626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:01:03.414137 30626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:01:03.414158 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.405072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.6.47.189:37419)
0810 23:01:03.405165 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.405168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.405177 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:01:03.405202 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.405203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.405212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.405277 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.405885 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.406576 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.407523 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.407824 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.407907 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.410903 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.410905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.410905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.410912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.412638 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.413063 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.413064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.413065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.413120 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.413318 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.413319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.413320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.413446 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.413607 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.413986 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.414063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 23:01:03.415561 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:01:03.415092 30458 master_runner.cc:186] Error getting master registration for 127.6.47.189:37419: OK, instance_id { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" instance_seqno: 1786402863213369 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:01:03.417413 30458 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:01:03.417601 30597 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:03.417728 30597 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.418031 30597 leader_election.cc:288] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71905b6b651b46fe931a5a2f38a8f91d (127.6.47.189:37419), 65c0cfffcfa643c1bc7dd3448710f0a8 (127.6.47.188:46821)
I20260810 23:01:03.418251 30525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0edd9e817124fe097e552f51214c61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" is_pre_election: true
I20260810 23:01:03.418430 30525 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0edd9e817124fe097e552f51214c61b in term 0.
I20260810 23:01:03.418646 30227 leader_election.cc:302] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c0cfffcfa643c1bc7dd3448710f0a8, e0edd9e817124fe097e552f51214c61b; no voters:
I20260810 23:01:03.418782 30597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:03.418859 30597 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:03.418895 30597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:03.419883 30597 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.420008 30597 leader_election.cc:288] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [CANDIDATE]: Term 1 election: Requested vote from peers 71905b6b651b46fe931a5a2f38a8f91d (127.6.47.189:37419), 65c0cfffcfa643c1bc7dd3448710f0a8 (127.6.47.188:46821)
I20260810 23:01:03.420351 30525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0edd9e817124fe097e552f51214c61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8"
I20260810 23:01:03.420436 30525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:03.421379 30525 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0edd9e817124fe097e552f51214c61b in term 1.
I20260810 23:01:03.421597 30227 leader_election.cc:302] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c0cfffcfa643c1bc7dd3448710f0a8, e0edd9e817124fe097e552f51214c61b; no voters:
I20260810 23:01:03.421710 30597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:03.421864 30597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [term 1 LEADER]: Becoming Leader. State: Replica: e0edd9e817124fe097e552f51214c61b, State: Running, Role: LEADER
I20260810 23:01:03.421962 30597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.422401 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0edd9e817124fe097e552f51214c61b. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.422497 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:03.427387 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.417421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.6.47.190:46753)
0810 23:01:03.418119 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.418121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.418308 (+ 187us) server_negotiation.cc:407] Connection header received
0810 23:01:03.418546 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.418547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.418559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.418590 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.419536 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.420267 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.421554 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.421835 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.421901 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.423940 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.423942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.423942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.423949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.425651 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.426304 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.426305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.426305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.426365 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.426681 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.426682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.426683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.426819 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.426894 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.427220 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.427302 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:01:03.427588 30648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:03.427850 30648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:03.433765 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.417057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.6.47.189:37419)
0810 23:01:03.417093 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.417095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.418128 (+ 1033us) server_negotiation.cc:407] Connection header received
0810 23:01:03.418425 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.418426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.418435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.418464 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.419923 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.420589 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.424609 (+ 4020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.424915 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.424961 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.429563 (+ 4602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.429565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.429566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.429573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.431215 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.431955 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.431956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.431957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.432103 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.432756 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.432757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.432758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.432942 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.433031 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.433582 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.433660 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:01:03.437491 30525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [term 1 FOLLOWER]: Refusing update from remote peer e0edd9e817124fe097e552f51214c61b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:03.437940 30646 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:03.438524 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.417884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.6.47.188:46821)
0810 23:01:03.417925 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.417926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.419680 (+ 1754us) server_negotiation.cc:407] Connection header received
0810 23:01:03.420729 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.420731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.420743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.420773 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.423421 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.424119 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.427757 (+ 3638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.428055 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.428105 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.434832 (+ 6727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.434834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.434834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.434842 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.436464 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.437257 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.437273 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.437273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.437337 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.437692 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.437693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.437694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.437831 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.437935 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.438337 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.438431 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:01:03.440629 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0edd9e817124fe097e552f51214c61b. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.440716 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.442165 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.442253 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:03.442373 30598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.442442 30598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.442442 30650 mvcc.cc:205] Tried to move back new op lower bound from 7317106128637296640 to 7317106128576995328. Current Snapshot: MvccSnapshot[applied={T|T < 7317106128637296640}]
I20260810 23:01:03.444988 30648 catalog_manager.cc:1347] Generated new cluster ID: f68331cb4a2a4a19b6ce39134a8000c8
I20260810 23:01:03.445044 30648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:01:03.449028 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } attempt: 3
Aug 10 23:01:03 dist-test-slave-rwrg krb5kdc[30174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
I20260810 23:01:03.458070 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.449281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:46821 (local address 127.0.0.1:55120)
0810 23:01:03.449328 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:03.449359 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:03.449412 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:03.449554 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:03.449555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:03.449757 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:03.449806 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.449808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.450649 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.450650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.451419 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:03.451422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.451946 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.451948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:03.452005 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.452562 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:03.452567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:03.454070 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:03.456143 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.456144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.456144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.456371 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.457365 (+ 994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:03.457366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:03.457366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:03.457437 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:03.457767 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:03.457768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:03.457826 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 23:01:03.457882 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 23:01:03.457974 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 23:01:03.458678 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:03.449384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.6.47.188:46821)
0810 23:01:03.449422 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:03.449424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:03.449456 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:01:03.449479 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:03.449479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:03.449488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:03.449549 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:03.449880 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.450581 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.451514 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:03.451827 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:03.451888 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:03.454212 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:03.454213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:03.454213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:03.454220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:03.456040 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.457188 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.457189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.457190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.457267 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:03.457530 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:03.457530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:03.457531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:03.457664 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:03.457725 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:03.458507 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 23:01:03.458583 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:01:03.460312 30447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: Bootstrap starting.
I20260810 23:01:03.461109 30447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:03.461452 30447 log.cc:824] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:03.470760 30447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: No bootstrap required, opened a new log
I20260810 23:01:03.472954 30447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.473104 30447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:03.473141 30447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71905b6b651b46fe931a5a2f38a8f91d, State: Initialized, Role: FOLLOWER
I20260810 23:01:03.473280 30447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } }
I20260810 23:01:03.473652 30447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:03.473681 30401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0edd9e817124fe097e552f51214c61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71905b6b651b46fe931a5a2f38a8f91d" is_pre_election: true
I20260810 23:01:03.473872 30399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0edd9e817124fe097e552f51214c61b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71905b6b651b46fe931a5a2f38a8f91d"
I20260810 23:01:03.474632 30401 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0edd9e817124fe097e552f51214c61b in term 0.
I20260810 23:01:03.474987 30398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:03.477227 30398 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [term 1 FOLLOWER]: Refusing update from remote peer e0edd9e817124fe097e552f51214c61b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:03.477524 30597 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [LEADER]: Connected to new peer: Peer: permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:03.477770 30648 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:01:03.478569 30648 catalog_manager.cc:1504] Loading token signing keys...
W20260810 23:01:03.482124 30674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:01:03.482183 30674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:01:03.475054 30657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.482319 30657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.482430 30657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0edd9e817124fe097e552f51214c61b. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.482491 30657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.482568 30657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.482610 30657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:01:03.487830 30597 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0edd9e817124fe097e552f51214c61b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0edd9e817124fe097e552f51214c61b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46753 } } peers { permanent_uuid: "71905b6b651b46fe931a5a2f38a8f91d" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 37419 } } peers { permanent_uuid: "65c0cfffcfa643c1bc7dd3448710f0a8" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 46821 } } }
I20260810 23:01:03.487962 30597 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:03.489593 30648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0edd9e817124fe097e552f51214c61b: Generated new TSK 0
I20260810 23:01:03.489841 30648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:01:03.490516 30676 mvcc.cc:205] Tried to move back new op lower bound from 7317106128794497024 to 7317106128576995328. Current Snapshot: MvccSnapshot[applied={T|T < 7317106128637296640}]
I20260810 23:01:04.414752 30626 catalog_manager.cc:1259] Loaded cluster ID: f68331cb4a2a4a19b6ce39134a8000c8
I20260810 23:01:04.414839 30626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: loading cluster ID for follower catalog manager: success
I20260810 23:01:04.415619 30626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: acquiring CA information for follower catalog manager: success
I20260810 23:01:04.415935 30626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65c0cfffcfa643c1bc7dd3448710f0a8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:01:04.482748 30674 catalog_manager.cc:1259] Loaded cluster ID: f68331cb4a2a4a19b6ce39134a8000c8
I20260810 23:01:04.482858 30674 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: loading cluster ID for follower catalog manager: success
I20260810 23:01:04.483637 30674 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: acquiring CA information for follower catalog manager: success
I20260810 23:01:04.483960 30674 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71905b6b651b46fe931a5a2f38a8f91d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:01:04.548272 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 30194
I20260810 23:01:04.557713 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 30338
I20260810 23:01:04.574385 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 30458
2026-08-10T23:01:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3054 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:01:04 dist-test-slave-rwrg krb5kdc[30713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:01:04 dist-test-slave-rwrg krb5kdc[30713](info): set up 2 sockets
Aug 10 23:01:04 dist-test-slave-rwrg krb5kdc[30713](info): commencing operation
krb5kdc: starting...
W20260810 23:01:05.892714 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:01:05 dist-test-slave-rwrg krb5kdc[30713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:01:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:01:06.017341 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42183
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:33115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:42183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:06.101137 30740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:06.101336 30740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:06.101357 30740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:06.102509 30740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:06.102553 30740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:06.102567 30740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:06.102607 30740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:06.102636 30740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:06.104140 30740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:33115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:42183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:06.104396 30740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:06.104665 30740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:06.109676 30757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:06.109730 30759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:06.109750 30756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:06.109983 30740 server_base.cc:1034] running on GCE node
I20260810 23:01:06.110255 30740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:06.110569 30740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:06.111753 30740 hybrid_clock.cc:648] HybridClock initialized: now 1786402866111745 us; error 36 us; skew 500 ppm
Aug 10 23:01:06 dist-test-slave-rwrg krb5kdc[30713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402866, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:06.121896 30740 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:06.122424 30740 webserver.cc:460] Webserver started at http://127.6.47.190:38055/ using document root <none> and password file <none>
I20260810 23:01:06.122670 30740 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:06.122743 30740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:06.122895 30740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:06.124004 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "b518ba618c6e4df89184a8f7a7169075"
format_stamp: "Formatted at 2026-08-10 23:01:06 on dist-test-slave-rwrg"
server_key: "9dadb508ee95cb1135c43f352fdefa18"
server_key_iv: "dd00ef27ec1623a12802cc58c50293f4"
server_key_version: "encryptionkey@0"
I20260810 23:01:06.124440 30740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "b518ba618c6e4df89184a8f7a7169075"
format_stamp: "Formatted at 2026-08-10 23:01:06 on dist-test-slave-rwrg"
server_key: "9dadb508ee95cb1135c43f352fdefa18"
server_key_iv: "dd00ef27ec1623a12802cc58c50293f4"
server_key_version: "encryptionkey@0"
I20260810 23:01:06.125985 30740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:01:06.126757 30769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:06.126950 30740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:06.127063 30740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "b518ba618c6e4df89184a8f7a7169075"
format_stamp: "Formatted at 2026-08-10 23:01:06 on dist-test-slave-rwrg"
server_key: "9dadb508ee95cb1135c43f352fdefa18"
server_key_iv: "dd00ef27ec1623a12802cc58c50293f4"
server_key_version: "encryptionkey@0"
I20260810 23:01:06.127136 30740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:06.140215 30740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:06.142138 30740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:06.142304 30740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:06.146409 30740 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:42183
I20260810 23:01:06.146486 30851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:42183 every 8 connection(s)
I20260810 23:01:06.146803 30740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:06.147665 30852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:06.150072 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075: Bootstrap starting.
I20260810 23:01:06.150674 30852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:06.150985 30852 log.cc:824] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:06.151714 30852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075: No bootstrap required, opened a new log
I20260810 23:01:06.153057 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 30740
I20260810 23:01:06.153147 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:01:06.153337 6334 external_mini_cluster.cc:1428] Setting key b7879f22c4bfe13b1fee151f05f4d032
I20260810 23:01:06.153601 30852 raft_consensus.cc:348] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } }
I20260810 23:01:06.153839 30852 raft_consensus.cc:374] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:06.153887 30852 raft_consensus.cc:729] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b518ba618c6e4df89184a8f7a7169075, State: Initialized, Role: FOLLOWER
I20260810 23:01:06.154023 30852 consensus_queue.cc:260] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } }
I20260810 23:01:06.154117 30852 raft_consensus.cc:388] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:06.154151 30852 raft_consensus.cc:482] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:06.154210 30852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:06.155005 30852 raft_consensus.cc:504] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } }
I20260810 23:01:06.155112 30852 leader_election.cc:302] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b518ba618c6e4df89184a8f7a7169075; no voters:
I20260810 23:01:06.155272 30852 leader_election.cc:288] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:01:06.155341 30860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:06.155478 30860 raft_consensus.cc:686] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [term 1 LEADER]: Becoming Leader. State: Replica: b518ba618c6e4df89184a8f7a7169075, State: Running, Role: LEADER
I20260810 23:01:06.155505 30852 sys_catalog.cc:564] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:06.155571 30860 consensus_queue.cc:237] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } }
I20260810 23:01:06.155910 30860 sys_catalog.cc:455] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b518ba618c6e4df89184a8f7a7169075. Latest consensus state: current_term: 1 leader_uuid: "b518ba618c6e4df89184a8f7a7169075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } } }
I20260810 23:01:06.155916 30863 sys_catalog.cc:455] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b518ba618c6e4df89184a8f7a7169075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b518ba618c6e4df89184a8f7a7169075" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 42183 } } }
I20260810 23:01:06.155997 30860 sys_catalog.cc:458] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:06.156003 30863 sys_catalog.cc:458] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:06.156782 30876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:06.157306 30876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:06.159754 30876 catalog_manager.cc:1347] Generated new cluster ID: 3670b8cddc934db7823d17f3bb5c861e
I20260810 23:01:06.159854 30876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:01:06 dist-test-slave-rwrg krb5kdc[30713](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:06.164790 30859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:06.153879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.6.47.190:42183)
0810 23:01:06.154077 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:06.154087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:06.154126 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:01:06.154263 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:06.154270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:06.154325 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:06.154463 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:06.155065 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:06.156509 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:06.157557 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:06.158146 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:06.158283 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:06.160896 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:06.160900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:06.160901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:06.160913 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:06.162980 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:06.163388 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:06.163391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:06.163391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:06.163516 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:06.163734 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:06.163735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:06.163736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:06.164123 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:06.164322 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:06.164393 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 23:01:06.164567 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260810 23:01:06.165822 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:01:06.175280 30876 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:01:06.175817 30876 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:01:06.181582 30876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b518ba618c6e4df89184a8f7a7169075: Generated new TSK 0
I20260810 23:01:06.181778 30876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:01:06.219869 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 30740
2026-08-10T23:01:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1644 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:01:06 dist-test-slave-rwrg krb5kdc[30896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:01:06 dist-test-slave-rwrg krb5kdc[30896](info): set up 2 sockets
Aug 10 23:01:06 dist-test-slave-rwrg krb5kdc[30896](info): commencing operation
krb5kdc: starting...
W20260810 23:01:07.504984 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:01:07 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:01:07.627027 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:07.712450 30920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:07.712774 30920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:07.712812 30920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:07.714277 30920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:07.714347 30920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:07.714375 30920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:07.714406 30920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:07.714432 30920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:07.715939 30920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:07.716224 30920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:07.716470 30920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:07.721266 30934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:07.721266 30936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:07.721302 30932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:07.721375 30920 server_base.cc:1034] running on GCE node
I20260810 23:01:07.721762 30920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:07.722083 30920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:07.723250 30920 hybrid_clock.cc:648] HybridClock initialized: now 1786402867723240 us; error 29 us; skew 500 ppm
Aug 10 23:01:07 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402867, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:07.725803 30920 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:07.726233 30920 webserver.cc:460] Webserver started at http://127.6.47.190:35807/ using document root <none> and password file <none>
I20260810 23:01:07.726483 30920 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:07.726527 30920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:07.726702 30920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:07.727773 30920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "2e64477a3fa741128f318bdcfa848260"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "8ee39e17f7ce6ce882115421cae62c6c"
server_key_iv: "487f32d1d90e9486c79746c5ab2610de"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.728127 30920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "2e64477a3fa741128f318bdcfa848260"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "8ee39e17f7ce6ce882115421cae62c6c"
server_key_iv: "487f32d1d90e9486c79746c5ab2610de"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.729660 30920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:01:07.730496 30947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:07.730715 30920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:07.730813 30920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2e64477a3fa741128f318bdcfa848260"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "8ee39e17f7ce6ce882115421cae62c6c"
server_key_iv: "487f32d1d90e9486c79746c5ab2610de"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.730868 30920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:07.745590 30920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:07.747903 30920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:07.748015 30920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:07.752128 30920 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:44331
I20260810 23:01:07.752188 31047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:44331 every 8 connection(s)
I20260810 23:01:07.752628 30920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:07.753640 31049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:07.756402 31049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap starting.
I20260810 23:01:07.757005 31049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:07.757264 31049 log.cc:824] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:07.757928 31049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: No bootstrap required, opened a new log
I20260810 23:01:07.759495 31049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:07.759649 31049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:07.759670 31049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Initialized, Role: FOLLOWER
I20260810 23:01:07.759795 31049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:07.759847 31049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:07.759868 31049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:07.759895 31049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:07.760696 31049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:07.760787 31049 leader_election.cc:302] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e64477a3fa741128f318bdcfa848260; no voters:
I20260810 23:01:07.760951 31049 leader_election.cc:288] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:01:07.760987 31057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:07.761121 31057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 LEADER]: Becoming Leader. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Running, Role: LEADER
I20260810 23:01:07.761232 31049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:07.761274 31057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:07.761747 31061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e64477a3fa741128f318bdcfa848260. Latest consensus state: current_term: 1 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:07.761747 31058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:07.761866 31061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:07.761871 31058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:07.762075 31074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:07.762606 31074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:07.762719 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 30920
I20260810 23:01:07.762826 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:01:07.763032 6334 external_mini_cluster.cc:1428] Setting key a4c9b43ddde446c2a83b7e0be0cc0646
I20260810 23:01:07.764607 31074 catalog_manager.cc:1347] Generated new cluster ID: 67a9eb0d44ed414ca347881e79eb57a5
I20260810 23:01:07.764662 31074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:01:07 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:07.774857 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:07.763512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.6.47.190:44331)
0810 23:01:07.763703 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:07.763713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:07.763762 (+ 49us) server_negotiation.cc:407] Connection header received
0810 23:01:07.763869 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:07.763875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:07.763938 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:07.764244 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:07.764895 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.766269 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.767429 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.767996 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.768095 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:07.771001 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:07.771006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:07.771006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:07.771020 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:07.773159 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:07.773548 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:07.773551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:07.773552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:07.773686 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:07.773895 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:07.773896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:07.773896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:07.774236 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:07.774395 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:07.774470 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 23:01:07.774614 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260810 23:01:07.784564 31074 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:01:07.785243 31074 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:01:07.788970 31074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Generated new TSK 0
I20260810 23:01:07.789165 31074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:01:07.816115 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:07.898928 31099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:07.899121 31099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:07.899144 31099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:07.900321 31099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:07.900364 31099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:07.900391 31099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:01:07.901942 31099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:07.902189 31099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:07.902463 31099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:07.907395 31110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:07.907449 31106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:07.907399 31107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:07.907699 31099 server_base.cc:1034] running on GCE node
I20260810 23:01:07.907889 31099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:07.908082 31099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:07.909216 31099 hybrid_clock.cc:648] HybridClock initialized: now 1786402867909204 us; error 28 us; skew 500 ppm
Aug 10 23:01:07 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402867, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:07.911885 31099 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:01:07.912372 31099 webserver.cc:460] Webserver started at http://127.6.47.129:43947/ using document root <none> and password file <none>
I20260810 23:01:07.912679 31099 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:07.912751 31099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:07.912909 31099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:07.913924 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "940ae1cefc6044db9703c37ed3f6db82"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "9ebd8e38a84e72f5d8bb77a3b2c20b79"
server_key_iv: "0f0fb8bcfefbf9137e28a07a901f0fc5"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.914289 31099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "940ae1cefc6044db9703c37ed3f6db82"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "9ebd8e38a84e72f5d8bb77a3b2c20b79"
server_key_iv: "0f0fb8bcfefbf9137e28a07a901f0fc5"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.915572 31099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:01:07.916285 31120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:07.916460 31099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:07.916585 31099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "940ae1cefc6044db9703c37ed3f6db82"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "9ebd8e38a84e72f5d8bb77a3b2c20b79"
server_key_iv: "0f0fb8bcfefbf9137e28a07a901f0fc5"
server_key_version: "encryptionkey@0"
I20260810 23:01:07.916679 31099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:07.924026 31099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:07.926347 31099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:07.926532 31099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:07.926774 31099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:07.927081 31099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:07.927136 31099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:07.927177 31099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:07.927205 31099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:07.933727 31099 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:37015
I20260810 23:01:07.933769 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:37015 every 8 connection(s)
I20260810 23:01:07.934110 31099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
Aug 10 23:01:07 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402867, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:07.941474 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 31099
I20260810 23:01:07.941612 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:01:07.941833 6334 external_mini_cluster.cc:1428] Setting key b497a412826458dff2915d8998e82153
I20260810 23:01:07.943868 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:07.934918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:39871 (local address 127.6.47.190:44331)
0810 23:01:07.934969 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:07.934971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:07.935188 (+ 217us) server_negotiation.cc:407] Connection header received
0810 23:01:07.935642 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:07.935643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:07.935653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:07.935707 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:07.936633 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.937126 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.938066 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.938267 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.938318 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:07.940646 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:07.940648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:07.940648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:07.940654 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:07.942230 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:07.942729 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:07.942730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:07.942730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:07.942782 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:07.943071 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:07.943072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:07.943072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:07.943178 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:07.943226 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:07.943615 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 23:01:07.943716 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:01:07.944443 31331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:07.934893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:39871)
0810 23:01:07.935050 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:07.935086 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:07.935557 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:07.935763 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:07.935766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:07.936348 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:07.936553 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:07.936558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.937211 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.937211 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:07.937992 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:07.937994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:07.938356 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:07.938357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:07.938476 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:07.939033 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:07.939041 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:07.940490 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:07.942320 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:07.942326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:07.942328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:07.942644 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:07.942853 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:07.942854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:07.942854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:07.943006 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:07.943240 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:07.943246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:07.943431 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 23:01:07.943970 (+ 539us) client_negotiation.cc:240] Negotiation successful
0810 23:01:07.944172 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260810 23:01:07.944974 31328 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:07.945087 31328 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:07.945338 31328 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:07.945848 30984 ts_manager.cc:194] Registered new tserver with Master: 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:07.946448 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:39871
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:01:07.968788 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:08.052289 31334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.052520 31334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.052546 31334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.053637 31334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.053679 31334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.053705 31334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:01:08.055086 31334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.055285 31334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.055465 31334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.060534 31348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.060539 31345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.060568 31342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.060644 31334 server_base.cc:1034] running on GCE node
I20260810 23:01:08.060956 31334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.061215 31334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.062381 31334 hybrid_clock.cc:648] HybridClock initialized: now 1786402868062340 us; error 73 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.064977 31334 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:01:08.065446 31334 webserver.cc:460] Webserver started at http://127.6.47.130:44991/ using document root <none> and password file <none>
I20260810 23:01:08.065627 31334 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.065673 31334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.065809 31334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:08.066902 31334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "7fc4ed5748b3497ea095b35076446748"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "100df16964ac4315c3e536a4f882b429"
server_key_iv: "67ffe3d8b1f10c01be4802d4c1771b8d"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.067384 31334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "7fc4ed5748b3497ea095b35076446748"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "100df16964ac4315c3e536a4f882b429"
server_key_iv: "67ffe3d8b1f10c01be4802d4c1771b8d"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.068807 31334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:01:08.069525 31358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.069716 31334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.069809 31334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "7fc4ed5748b3497ea095b35076446748"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "100df16964ac4315c3e536a4f882b429"
server_key_iv: "67ffe3d8b1f10c01be4802d4c1771b8d"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.069882 31334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.082508 31334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.084378 31334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.084569 31334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.084817 31334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:08.085165 31334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:08.085199 31334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.085222 31334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:08.085268 31334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.091531 31334 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40611
I20260810 23:01:08.091634 31560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40611 every 8 connection(s)
I20260810 23:01:08.091938 31334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:01:08.094311 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 31334
I20260810 23:01:08.094434 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:01:08.094664 6334 external_mini_cluster.cc:1428] Setting key 3a27db434e86693fe9cf1c8ed2a89e03
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:08.102146 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.092715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:42395)
0810 23:01:08.092899 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.092937 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.093453 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.093739 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.093743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.094319 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:08.094622 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.094628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.095448 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.095449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.096272 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.096274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.096735 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.096737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.096882 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.097394 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:08.097402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:08.098993 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:08.100616 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.100624 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.100626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.100967 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.101190 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.101191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.101191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.101338 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.101609 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:08.101615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:08.101774 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.101864 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.101985 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260810 23:01:08.102177 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.092732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:42395 (local address 127.6.47.190:44331)
0810 23:01:08.092778 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.092781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.093033 (+ 252us) server_negotiation.cc:407] Connection header received
0810 23:01:08.093570 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.093571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.093585 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.093640 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.094718 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.095349 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.096365 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.096650 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.096726 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.099078 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.099080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.099080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.099087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.100485 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.101045 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.101046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.101046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.101096 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.101411 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.101411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.101412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.101511 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.101599 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.101921 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.102025 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:01:08.102784 31561 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:08.102870 31561 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:08.103068 31561 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:08.103521 30984 ts_manager.cc:194] Registered new tserver with Master: 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
I20260810 23:01:08.104060 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:42395
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:01:08.124039 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:08.208148 31567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.208357 31567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.208379 31567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.209623 31567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.209666 31567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.209692 31567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:01:08.211061 31567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.211261 31567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.211486 31567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.216627 31575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.216708 31582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.216627 31577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.216723 31567 server_base.cc:1034] running on GCE node
I20260810 23:01:08.217097 31567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.217332 31567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.218498 31567 hybrid_clock.cc:648] HybridClock initialized: now 1786402868218470 us; error 58 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.221046 31567 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:01:08.221597 31567 webserver.cc:460] Webserver started at http://127.6.47.131:32783/ using document root <none> and password file <none>
I20260810 23:01:08.221828 31567 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.221897 31567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.222056 31567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:08.223148 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "94f07487235246218c0b9b9254494013"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "c451d1b35129260cd0c5b0603946f608"
server_key_iv: "2dd84f8944697a49dc58f265c05649a6"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.223562 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "94f07487235246218c0b9b9254494013"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "c451d1b35129260cd0c5b0603946f608"
server_key_iv: "2dd84f8944697a49dc58f265c05649a6"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.224994 31567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:01:08.225745 31594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.225919 31567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:01:08.226030 31567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "94f07487235246218c0b9b9254494013"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "c451d1b35129260cd0c5b0603946f608"
server_key_iv: "2dd84f8944697a49dc58f265c05649a6"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.226121 31567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.235471 31567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.237366 31567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.237522 31567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.237737 31567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:08.238029 31567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:08.238090 31567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.238128 31567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:08.238157 31567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.244239 31567 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:44171
I20260810 23:01:08.244297 31801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:44171 every 8 connection(s)
I20260810 23:01:08.244675 31567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:01:08.249923 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 31567
I20260810 23:01:08.250003 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:01:08.250191 6334 external_mini_cluster.cc:1428] Setting key ee7bfb997b030c26faef9a4a136cdc22
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:08.254194 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.245369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:38317 (local address 127.6.47.190:44331)
0810 23:01:08.245414 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.245417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.245618 (+ 201us) server_negotiation.cc:407] Connection header received
0810 23:01:08.246114 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.246115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.246125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.246185 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.247123 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.247591 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.248678 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.248851 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.248904 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.251235 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.251237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.251237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.251244 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.252770 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.253224 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.253226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.253226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.253284 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.253546 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.253547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.253547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.253659 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.253754 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.254026 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.254103 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:01:08.254395 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.245314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:38317)
0810 23:01:08.245470 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.245512 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.246019 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.246260 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.246263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.246858 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:08.247043 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.247048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.247670 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.247671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.248451 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.248453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.248944 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.248945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.249086 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.249577 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:08.249585 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:08.251157 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:08.252887 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.252896 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.252898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.253125 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.253361 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.253362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.253363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.253472 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.253758 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:08.253766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:08.253899 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.254002 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.254154 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260810 23:01:08.254814 31802 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:08.254914 31802 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:08.255090 31802 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:08.255470 30984 ts_manager.cc:194] Registered new tserver with Master: 94f07487235246218c0b9b9254494013 (127.6.47.131:44171)
I20260810 23:01:08.255836 30984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:38317
I20260810 23:01:08.262223 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:01:08.269866 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.263711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34382 (local address 127.6.47.190:44331)
0810 23:01:08.263768 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.263769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.263900 (+ 131us) server_negotiation.cc:407] Connection header received
0810 23:01:08.263953 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.263953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.263962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.264002 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.264600 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.265054 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.265762 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.265960 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.266031 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.267191 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.267192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.267193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.267200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.268639 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.269005 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.269006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.269007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.269068 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.269295 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.269296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.269297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.269445 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.269566 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.269667 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.269746 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:01:08.271488 30984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:01:08.273132 30984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:01:08.279666 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40611 (local address 127.0.0.1:35024)
0810 23:01:08.275748 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.275795 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.275866 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.276215 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.276218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.276221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:08.276292 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.276294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277633 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.277634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.278672 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.278674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279329 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.279411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279424 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.279523 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.279581 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260810 23:01:08.279675 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.6.47.130:40611)
0810 23:01:08.275958 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.275961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.275971 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:01:08.276015 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.276017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.276029 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.276141 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:08.276380 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277517 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.278758 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279242 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279327 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279385 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.279543 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.279583 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":92,"threads_started":1}
I20260810 23:01:08.279819 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44171 (local address 127.0.0.1:44422)
0810 23:01:08.275995 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.276005 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.276066 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.276330 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.276331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.276333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:08.276390 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.276391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277277 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.277278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.278482 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.278485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279415 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.279479 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279483 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.279537 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.279582 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":23,"thread_start_us":60,"threads_started":1}
I20260810 23:01:08.279978 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.6.47.129:37015)
0810 23:01:08.275981 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.275987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.276368 (+ 381us) server_negotiation.cc:407] Connection header received
0810 23:01:08.276415 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.276418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.276434 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.276535 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:08.276626 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277595 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.278814 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279510 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279567 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279619 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.279866 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.279906 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":120,"threads_started":1}
I20260810 23:01:08.280166 31819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.0.0.1:42220)
0810 23:01:08.276240 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.276251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.276303 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.276528 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.276529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.276530 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:08.276561 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.276562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277696 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.277697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.278719 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.278721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279655 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.279707 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279710 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.279762 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.279807 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":36,"thread_start_us":43,"threads_started":1}
I20260810 23:01:08.280367 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.275644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.6.47.131:44171)
0810 23:01:08.275848 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.275854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.276111 (+ 257us) server_negotiation.cc:407] Connection header received
0810 23:01:08.276155 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.276158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.276174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.276250 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:08.276448 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.277174 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.278571 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.279252 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.279320 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.279375 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.279647 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.279690 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260810 23:01:08.280529 31436 tablet_service.cc:1467] Processing CreateTablet for tablet 14b534caeaa64480ad9321fd0895e558 (DEFAULT_TABLE table=test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:08.280942 31436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b534caeaa64480ad9321fd0895e558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:08.281028 31205 tablet_service.cc:1467] Processing CreateTablet for tablet 14b534caeaa64480ad9321fd0895e558 (DEFAULT_TABLE table=test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:08.281220 31678 tablet_service.cc:1467] Processing CreateTablet for tablet 14b534caeaa64480ad9321fd0895e558 (DEFAULT_TABLE table=test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:08.281399 31205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b534caeaa64480ad9321fd0895e558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:08.281579 31678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b534caeaa64480ad9321fd0895e558. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:08.283962 31826 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap starting.
I20260810 23:01:08.284474 31827 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Bootstrap starting.
I20260810 23:01:08.284798 31826 tablet_bootstrap.cc:654] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:08.285187 31830 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Bootstrap starting.
I20260810 23:01:08.285218 31827 tablet_bootstrap.cc:654] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:08.285632 31826 log.cc:824] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:08.285770 31827 log.cc:824] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:08.285841 31830 tablet_bootstrap.cc:654] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:08.286392 31826 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: No bootstrap required, opened a new log
I20260810 23:01:08.286417 31830 log.cc:824] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:08.286448 31826 ts_tablet_manager.cc:1397] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:01:08.286648 31827 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: No bootstrap required, opened a new log
I20260810 23:01:08.286727 31827 ts_tablet_manager.cc:1397] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:01:08.287209 31830 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: No bootstrap required, opened a new log
I20260810 23:01:08.287281 31830 ts_tablet_manager.cc:1397] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:01:08.288173 31826 raft_consensus.cc:348] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.288389 31826 raft_consensus.cc:374] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:08.288439 31826 raft_consensus.cc:729] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940ae1cefc6044db9703c37ed3f6db82, State: Initialized, Role: FOLLOWER
I20260810 23:01:08.288576 31826 consensus_queue.cc:260] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.288885 31826 ts_tablet_manager.cc:1428] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:01:08.288885 31328 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
I20260810 23:01:08.288851 31827 raft_consensus.cc:348] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.289034 31827 raft_consensus.cc:374] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:08.289079 31827 raft_consensus.cc:729] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc4ed5748b3497ea095b35076446748, State: Initialized, Role: FOLLOWER
I20260810 23:01:08.289216 31827 consensus_queue.cc:260] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.289278 31830 raft_consensus.cc:348] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.289472 31830 raft_consensus.cc:374] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:08.289513 31830 raft_consensus.cc:729] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f07487235246218c0b9b9254494013, State: Initialized, Role: FOLLOWER
I20260810 23:01:08.289608 31830 consensus_queue.cc:260] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.289770 31561 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
I20260810 23:01:08.289894 31830 ts_tablet_manager.cc:1428] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:01:08.289927 31802 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
I20260810 23:01:08.290302 31827 ts_tablet_manager.cc:1428] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260810 23:01:08.320950 31840 raft_consensus.cc:482] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:08.321112 31840 raft_consensus.cc:504] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.321503 31840 leader_election.cc:288] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611), 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:08.325438 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.321827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:42937 (local address 127.6.47.130:40611)
0810 23:01:08.321877 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.321879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.321887 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:01:08.321915 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.321916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.321929 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.321986 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:08.322202 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.322978 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.324413 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.325055 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.325124 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.325133 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.325321 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.325362 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:01:08.325438 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.321978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:53839 (local address 127.6.47.129:37015)
0810 23:01:08.322086 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.322090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.322103 (+ 13us) server_negotiation.cc:407] Connection header received
0810 23:01:08.322139 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.322139 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.322150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.322220 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:08.322365 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.323049 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.324413 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.325038 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.325124 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.325133 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.325321 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.325362 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 23:01:08.325461 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.321714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40611 (local address 127.6.47.131:42937)
0810 23:01:08.321773 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.321785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.321832 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.322038 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.322039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.322041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:08.322123 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.322124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.323059 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.323060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.324300 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.324303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.325131 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.325132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.325232 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.325238 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.325299 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.325352 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":17}
I20260810 23:01:08.325477 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.321843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.6.47.131:53839)
0810 23:01:08.322039 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.322050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.322096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.322239 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.322240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.322241 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:08.322293 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.322294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.323126 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.323127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.324300 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.324303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.325131 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.325132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.325232 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.325238 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.325299 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.325352 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":10,"thread_start_us":124,"threads_started":1}
I20260810 23:01:08.325863 31241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14b534caeaa64480ad9321fd0895e558" candidate_uuid: "94f07487235246218c0b9b9254494013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940ae1cefc6044db9703c37ed3f6db82" is_pre_election: true
I20260810 23:01:08.325883 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14b534caeaa64480ad9321fd0895e558" candidate_uuid: "94f07487235246218c0b9b9254494013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc4ed5748b3497ea095b35076446748" is_pre_election: true
I20260810 23:01:08.326030 31241 raft_consensus.cc:2454] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f07487235246218c0b9b9254494013 in term 0.
I20260810 23:01:08.326042 31474 raft_consensus.cc:2454] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f07487235246218c0b9b9254494013 in term 0.
I20260810 23:01:08.326305 31595 leader_election.cc:302] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82, 94f07487235246218c0b9b9254494013; no voters:
I20260810 23:01:08.326467 31840 raft_consensus.cc:2781] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:08.326574 31840 raft_consensus.cc:482] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:08.326620 31840 raft_consensus.cc:3037] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:08.327412 31840 raft_consensus.cc:504] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.327567 31840 leader_election.cc:288] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [CANDIDATE]: Term 1 election: Requested vote from peers 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611), 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:08.327781 31474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14b534caeaa64480ad9321fd0895e558" candidate_uuid: "94f07487235246218c0b9b9254494013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc4ed5748b3497ea095b35076446748"
I20260810 23:01:08.327818 31241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14b534caeaa64480ad9321fd0895e558" candidate_uuid: "94f07487235246218c0b9b9254494013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940ae1cefc6044db9703c37ed3f6db82"
I20260810 23:01:08.327883 31474 raft_consensus.cc:3037] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:08.327886 31241 raft_consensus.cc:3037] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:08.328716 31241 raft_consensus.cc:2454] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f07487235246218c0b9b9254494013 in term 1.
I20260810 23:01:08.328840 31474 raft_consensus.cc:2454] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f07487235246218c0b9b9254494013 in term 1.
I20260810 23:01:08.328933 31595 leader_election.cc:302] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82, 94f07487235246218c0b9b9254494013; no voters:
I20260810 23:01:08.329057 31840 raft_consensus.cc:2781] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:08.329264 31840 raft_consensus.cc:686] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 1 LEADER]: Becoming Leader. State: Replica: 94f07487235246218c0b9b9254494013, State: Running, Role: LEADER
I20260810 23:01:08.329375 31840 consensus_queue.cc:237] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.330345 30984 catalog_manager.cc:5495] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94f07487235246218c0b9b9254494013 (127.6.47.131). New cstate: current_term: 1 leader_uuid: "94f07487235246218c0b9b9254494013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } health_report { overall_health: UNKNOWN } } }
I20260810 23:01:08.338085 31824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.335773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.6.47.131:44171)
0810 23:01:08.335831 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.335833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.335843 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:01:08.335882 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.335883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.335899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.335970 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:08.336095 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.336634 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.337353 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.337542 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.337602 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.337741 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:08.337841 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:08.337903 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.337934 (+ 31us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.337951 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:01:08.339887 31474 raft_consensus.cc:1261] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Refusing update from remote peer 94f07487235246218c0b9b9254494013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:08.339887 31241 raft_consensus.cc:1261] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Refusing update from remote peer 94f07487235246218c0b9b9254494013: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:08.340265 31843 consensus_queue.cc:1035] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:08.340404 31840 consensus_queue.cc:1035] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 23:01:08.342468 31563 tablet.cc:2369] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:01:08.343628 31849 mvcc.cc:205] Tried to move back new op lower bound from 7317106148717281280 to 7317106148677894144. Current Snapshot: MvccSnapshot[applied={T|T < 7317106148717281280}]
I20260810 23:01:08.343739 31852 mvcc.cc:205] Tried to move back new op lower bound from 7317106148717281280 to 7317106148677894144. Current Snapshot: MvccSnapshot[applied={T|T < 7317106148717281280}]
I20260810 23:01:08.344221 31851 mvcc.cc:205] Tried to move back new op lower bound from 7317106148717281280 to 7317106148677894144. Current Snapshot: MvccSnapshot[applied={T|T < 7317106148717281280}]
I20260810 23:01:08.346331 30981 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:08.346457 30981 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:08.347293 30981 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Sending DeleteTablet for 3 replicas of tablet 14b534caeaa64480ad9321fd0895e558
I20260810 23:01:08.347649 31678 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b534caeaa64480ad9321fd0895e558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:08 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:44422
I20260810 23:01:08.347719 31436 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b534caeaa64480ad9321fd0895e558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:08 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:35024
I20260810 23:01:08.347750 31205 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b534caeaa64480ad9321fd0895e558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:08 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42220
I20260810 23:01:08.347899 31867 tablet_replica.cc:330] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: stopping tablet replica
I20260810 23:01:08.347997 31867 raft_consensus.cc:2229] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:01:08.348062 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 31099
I20260810 23:01:08.348120 31867 raft_consensus.cc:2258] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:08.355345 31867 ts_tablet_manager.cc:1905] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:08.357834 31867 ts_tablet_manager.cc:1918] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:08.357932 31867 log.cc:1196] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/14b534caeaa64480ad9321fd0895e558
I20260810 23:01:08.358283 31867 ts_tablet_manager.cc:1939] T 14b534caeaa64480ad9321fd0895e558 P 94f07487235246218c0b9b9254494013: Deleting consensus metadata
I20260810 23:01:08.358911 30951 catalog_manager.cc:4841] TS 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): tablet 14b534caeaa64480ad9321fd0895e558 (table test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]) successfully deleted
I20260810 23:01:08.363296 31868 tablet_replica.cc:330] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: stopping tablet replica
I20260810 23:01:08.363471 31868 raft_consensus.cc:2229] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:08.363626 31868 raft_consensus.cc:2258] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:08.364918 31868 ts_tablet_manager.cc:1905] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:08.366588 31868 ts_tablet_manager.cc:1918] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:08.366674 31868 log.cc:1196] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/14b534caeaa64480ad9321fd0895e558
I20260810 23:01:08.366971 31868 ts_tablet_manager.cc:1939] T 14b534caeaa64480ad9321fd0895e558 P 7fc4ed5748b3497ea095b35076446748: Deleting consensus metadata
I20260810 23:01:08.367514 30950 catalog_manager.cc:4841] TS 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): tablet 14b534caeaa64480ad9321fd0895e558 (table test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]) successfully deleted
W20260810 23:01:08.369529 30950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:37015 (error 108)
I20260810 23:01:08.370114 31819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.369957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.0.0.1:42234)
0810 23:01:08.370027 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.370069 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 23:01:08.370164 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 31334
W20260810 23:01:08.370292 30950 catalog_manager.cc:4584] TS 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14b534caeaa64480ad9321fd0895e558: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37015: connect: Connection refused (error 111)
I20260810 23:01:08.390893 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 31567
I20260810 23:01:08.403149 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 30920
I20260810 23:01:08.410595 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--webserver_interface=127.6.47.190
--webserver_port=35807
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:08.493517 31873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.493806 31873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.493847 31873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.494982 31873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:08.495064 31873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.495090 31873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.495111 31873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:08.495141 31873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:08.496647 31873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=35807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.496930 31873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.497160 31873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.506815 31886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.506824 31885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.506839 31873 server_base.cc:1034] running on GCE node
W20260810 23:01:08.506851 31888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.507346 31873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.507696 31873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.508863 31873 hybrid_clock.cc:648] HybridClock initialized: now 1786402868508851 us; error 33 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.511574 31873 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:08.512025 31873 webserver.cc:460] Webserver started at http://127.6.47.190:35807/ using document root <none> and password file <none>
I20260810 23:01:08.512292 31873 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.512379 31873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.513899 31873 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:08.514634 31899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.514824 31873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:08.514935 31873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2e64477a3fa741128f318bdcfa848260"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "8ee39e17f7ce6ce882115421cae62c6c"
server_key_iv: "487f32d1d90e9486c79746c5ab2610de"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.515357 31873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.536819 31873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.539261 31873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.539429 31873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.543776 31873 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:44331
I20260810 23:01:08.543813 31988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:44331 every 8 connection(s)
I20260810 23:01:08.544283 31873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:08.546005 31989 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:01:08.546386 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 31873
I20260810 23:01:08.546706 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:37015
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=43947
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:08.547005 31989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap starting.
I20260810 23:01:08.558761 31989 log.cc:824] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:08.561532 31989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:08.561899 31989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap complete.
I20260810 23:01:08.565222 31989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:08.565522 31989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Initialized, Role: FOLLOWER
I20260810 23:01:08.565685 31989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:08.565749 31989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:08.565791 31989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:08.565837 31989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:01:08.567095 31989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:08.567207 31989 leader_election.cc:302] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e64477a3fa741128f318bdcfa848260; no voters:
I20260810 23:01:08.567378 31989 leader_election.cc:288] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:01:08.567468 31995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:01:08.567638 31989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:08.567660 31995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 LEADER]: Becoming Leader. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Running, Role: LEADER
I20260810 23:01:08.567883 31995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:08.568205 31999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:08.568285 31999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:08.568734 32017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:08.569583 32017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:08.569716 32017 catalog_manager.cc:1259] Loaded cluster ID: 67a9eb0d44ed414ca347881e79eb57a5
I20260810 23:01:08.569761 32001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e64477a3fa741128f318bdcfa848260. Latest consensus state: current_term: 2 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:08.569936 32017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:01:08.569950 32001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:08.570186 32017 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 23:01:08.575234 32017 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 23:01:08.577814 32017 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:01:08.578013 32017 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Loaded TSK: 0
I20260810 23:01:08.578227 32017 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:01:08.640010 31993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.640213 31993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.640236 31993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.641435 31993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.641480 31993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.641511 31993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:01:08.642983 31993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:37015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=43947
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.643253 31993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.643515 31993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.648679 32042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.648717 32039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.648731 32045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.649019 31993 server_base.cc:1034] running on GCE node
I20260810 23:01:08.649219 31993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.649494 31993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.650650 31993 hybrid_clock.cc:648] HybridClock initialized: now 1786402868650630 us; error 39 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.653414 31993 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:01:08.653852 31993 webserver.cc:460] Webserver started at http://127.6.47.129:43947/ using document root <none> and password file <none>
I20260810 23:01:08.654102 31993 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.654194 31993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.655593 31993 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:08.656358 32058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.656566 31993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:08.656673 31993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "940ae1cefc6044db9703c37ed3f6db82"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "9ebd8e38a84e72f5d8bb77a3b2c20b79"
server_key_iv: "0f0fb8bcfefbf9137e28a07a901f0fc5"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.657037 31993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.695048 31993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.697163 31993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.697340 31993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.697613 31993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:08.698064 32080 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:01:08.699623 31993 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:01:08.699687 31993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:01:08.699735 31993 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:01:08.700426 31993 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:01:08.700482 31993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:01:08.700588 32080 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap starting.
I20260810 23:01:08.707858 31993 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:37015
I20260810 23:01:08.707966 32273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:37015 every 8 connection(s)
I20260810 23:01:08.708302 31993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:01:08.711565 32080 log.cc:824] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:08.713812 32080 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:08.714155 32080 tablet_bootstrap.cc:492] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap complete.
I20260810 23:01:08.714218 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 31993
I20260810 23:01:08.714527 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:40611
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=44991
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:08.715426 32080 ts_tablet_manager.cc:1397] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:08.716813 32080 raft_consensus.cc:348] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.717164 32080 raft_consensus.cc:729] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940ae1cefc6044db9703c37ed3f6db82, State: Initialized, Role: FOLLOWER
I20260810 23:01:08.717357 32080 consensus_queue.cc:260] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:08.717653 32080 ts_tablet_manager.cc:1428] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260810 23:01:08.720387 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.709271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:34351 (local address 127.6.47.190:44331)
0810 23:01:08.709488 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.709496 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.709538 (+ 42us) server_negotiation.cc:407] Connection header received
0810 23:01:08.710097 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.710102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.710149 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.710295 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.710935 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.711907 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.713072 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.713470 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.713598 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.716470 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.716475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.716475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.716488 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.718519 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.719026 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.719029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.719029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.719183 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.719432 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.719433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.719433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.719661 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.719754 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.720068 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.720207 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
I20260810 23:01:08.720386 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.709067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:34351)
0810 23:01:08.709295 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.709340 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.709863 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.710325 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.710329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.710627 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:08.710809 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.710814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.712044 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.712049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.712978 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.712982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.713593 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.713595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.713750 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.714409 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:08.714420 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:08.716339 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:08.718649 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.718656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.718659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.718911 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.719247 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.719247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.719248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.719355 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.719746 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:08.719753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:08.719893 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.719998 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.720193 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
I20260810 23:01:08.721036 32274 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:08.721131 32274 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:08.721310 32274 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:08.722085 31929 ts_manager.cc:194] Registered new tserver with Master: 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:08.723198 31929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:34351
I20260810 23:01:08.724107 32274 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.129@KRBTEST.COM
I20260810 23:01:08.731603 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.722812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.6.47.129:37015)
0810 23:01:08.722959 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.722963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.723092 (+ 129us) server_negotiation.cc:407] Connection header received
0810 23:01:08.723198 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.723200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.723216 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.723295 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.723967 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.724843 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.726084 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.726359 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.726415 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.728619 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.728627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.728627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.728635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.730202 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.730633 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.730636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.730636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.730706 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.730922 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.730923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.730924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.731108 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.731185 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.731413 (+ 228us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.731505 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260810 23:01:08.731709 32286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.722732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.0.0.1:42244)
0810 23:01:08.722979 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.723014 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.723097 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.723399 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.723424 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.723794 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:08.723865 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.723868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.724927 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.724928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.725995 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.725999 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.726445 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.726446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.726543 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.727032 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:08.727038 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:08.728475 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:08.730310 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.730318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.730320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.730529 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.730785 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.730786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.730786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.730850 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.731183 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:08.731218 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:08.731276 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.731362 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.731491 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
I20260810 23:01:08.732196 32162 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b534caeaa64480ad9321fd0895e558 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:08 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42244
I20260810 23:01:08.732760 32289 tablet_replica.cc:330] stopping tablet replica
I20260810 23:01:08.732874 32289 raft_consensus.cc:2229] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:08.732946 32289 raft_consensus.cc:2258] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:08.733263 32289 ts_tablet_manager.cc:1905] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:08.735587 32289 ts_tablet_manager.cc:1918] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:08.735675 32289 log.cc:1196] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/14b534caeaa64480ad9321fd0895e558
I20260810 23:01:08.735924 32289 ts_tablet_manager.cc:1939] T 14b534caeaa64480ad9321fd0895e558 P 940ae1cefc6044db9703c37ed3f6db82: Deleting consensus metadata
I20260810 23:01:08.736477 31900 catalog_manager.cc:4841] TS 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015): tablet 14b534caeaa64480ad9321fd0895e558 (table test-table [id=cbd2e91e26f342c395c30f8fa0c062fd]) successfully deleted
W20260810 23:01:08.803298 32281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.803504 32281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.803527 32281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.804690 32281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.804730 32281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.804757 32281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:01:08.806192 32281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:40611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=44991
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.806401 32281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.806565 32281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.811801 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.811833 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.811815 32303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.812016 32281 server_base.cc:1034] running on GCE node
I20260810 23:01:08.812237 32281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.812552 32281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.813717 32281 hybrid_clock.cc:648] HybridClock initialized: now 1786402868813693 us; error 52 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.816341 32281 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:01:08.816839 32281 webserver.cc:460] Webserver started at http://127.6.47.130:44991/ using document root <none> and password file <none>
I20260810 23:01:08.817087 32281 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.817165 32281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.818696 32281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:01:08.819355 32317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.819545 32281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.819644 32281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "7fc4ed5748b3497ea095b35076446748"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "100df16964ac4315c3e536a4f882b429"
server_key_iv: "67ffe3d8b1f10c01be4802d4c1771b8d"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.819998 32281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.852440 32281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.854704 32281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.854876 32281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.855144 32281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:08.855500 32281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:08.855536 32281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.855584 32281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:08.855613 32281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.862448 32281 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40611
I20260810 23:01:08.862500 32539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40611 every 8 connection(s)
I20260810 23:01:08.862905 32281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:08.871649 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 32281
I20260810 23:01:08.871912 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:44171
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=32783
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:08.873217 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.863666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:37995 (local address 127.6.47.190:44331)
0810 23:01:08.863736 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:08.863762 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:08.863977 (+ 215us) server_negotiation.cc:407] Connection header received
0810 23:01:08.864614 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:08.864615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:08.864634 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:08.864691 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:08.865768 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.866333 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.867485 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.867803 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.867895 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.870212 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:08.870213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:08.870214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:08.870222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:08.871502 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.872073 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.872075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.872075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.872148 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:08.872413 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:08.872414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:08.872414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:08.872612 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:08.872700 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:08.873002 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 23:01:08.873105 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:01:08.873416 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.863631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:37995)
0810 23:01:08.863830 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:08.863879 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:08.864458 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:08.864794 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:08.864799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:08.865435 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:08.865642 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.865647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.866455 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.866456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.867380 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:08.867383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:08.867906 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:08.867908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:08.868047 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:08.868573 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:08.868582 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:08.870096 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:08.871614 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.871622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.871624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.871938 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.872234 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:08.872234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:08.872234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:08.872336 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:08.872692 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:08.872702 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:08.872852 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 23:01:08.872978 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 23:01:08.873146 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260810 23:01:08.874017 32540 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:08.874141 32540 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:08.874338 32540 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:08.874778 31929 ts_manager.cc:194] Registered new tserver with Master: 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
I20260810 23:01:08.875147 31929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:37995
W20260810 23:01:08.959326 32546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:08.959517 32546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:08.959540 32546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:08.960835 32546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:08.960880 32546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:08.960932 32546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:01:08.962554 32546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:44171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=32783
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:08.962802 32546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:08.963023 32546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:08.968138 32559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.968148 32561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:08.968127 32563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:08.968564 32546 server_base.cc:1034] running on GCE node
I20260810 23:01:08.968791 32546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:08.969084 32546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:08.970228 32546 hybrid_clock.cc:648] HybridClock initialized: now 1786402868970208 us; error 37 us; skew 500 ppm
Aug 10 23:01:08 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:08.972751 32546 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:01:08.973232 32546 webserver.cc:460] Webserver started at http://127.6.47.131:32783/ using document root <none> and password file <none>
I20260810 23:01:08.973484 32546 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:08.973562 32546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:08.974922 32546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:08.975484 32578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.975667 32546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:01:08.975733 32546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "94f07487235246218c0b9b9254494013"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "c451d1b35129260cd0c5b0603946f608"
server_key_iv: "2dd84f8944697a49dc58f265c05649a6"
server_key_version: "encryptionkey@0"
I20260810 23:01:08.976065 32546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:08.989948 32546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:08.991791 32546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:08.991958 32546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:08.992210 32546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:08.992590 32546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:08.992643 32546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.992677 32546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:08.992707 32546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:08.998746 32546 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:44171
I20260810 23:01:08.998785 330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:44171 every 8 connection(s)
I20260810 23:01:08.999164 32546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:01:09 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:09.008464 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.999947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:39003 (local address 127.6.47.190:44331)
0810 23:01:09.000009 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.000011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.000222 (+ 211us) server_negotiation.cc:407] Connection header received
0810 23:01:09.000790 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.000791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.000801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.000861 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:09.001810 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.002293 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.003276 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.003457 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.003509 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.005726 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:09.005728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:09.005728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:09.005735 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:09.007204 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:09.007557 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:09.007558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:09.007558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:09.007611 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:09.007824 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:09.007824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:09.007824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:09.007929 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:09.007979 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.008296 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.008366 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:01:09.008656 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 32546
I20260810 23:01:09.008714 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:08.999909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:39003)
0810 23:01:09.000083 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.000128 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.000701 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.000929 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:09.000933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:09.001499 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:09.001731 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.001736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.002375 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.002376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.003198 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.003201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.003532 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.003533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.003634 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.004121 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:09.004129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:09.005625 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:09.007278 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:09.007284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:09.007285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:09.007492 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:09.007670 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:09.007671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:09.007672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:09.007768 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:09.007997 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:09.008003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:09.008175 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 23:01:09.008266 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 23:01:09.008390 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260810 23:01:09.009513 331 heartbeater.cc:347] Connected to a master server at 127.6.47.190:44331
I20260810 23:01:09.009625 331 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:09.009833 331 heartbeater.cc:511] Master 127.6.47.190:44331 requested a full tablet report, sending...
I20260810 23:01:09.010289 31929 ts_manager.cc:194] Registered new tserver with Master: 94f07487235246218c0b9b9254494013 (127.6.47.131:44171)
I20260810 23:01:09.010676 31929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:39003
I20260810 23:01:09.015194 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.009116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34392 (local address 127.6.47.190:44331)
0810 23:01:09.009160 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.009162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.009341 (+ 179us) server_negotiation.cc:407] Connection header received
0810 23:01:09.009432 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.009433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.009444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.009525 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:09.010308 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.010883 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.011870 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.012044 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.012097 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.013206 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:09.013207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:09.013208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:09.013214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:09.014340 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:09.014625 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:09.014626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:09.014626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:09.014670 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:09.014821 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:09.014822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:09.014822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:09.014902 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:09.015010 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.015055 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.015111 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":21}
I20260810 23:01:09.015928 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:01:09.018509 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.016250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.6.47.190:44331)
0810 23:01:09.016286 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.016288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.016378 (+ 90us) server_negotiation.cc:407] Connection header received
0810 23:01:09.016450 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.016451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.016479 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.016617 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:09.016699 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.017106 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.017851 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.018058 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.018109 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.018244 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:09.018334 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:09.018398 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.018410 (+ 12us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.018424 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:01:09.018805 31929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:01:09.020421 31929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:01:09.022738 32162 tablet_service.cc:1467] Processing CreateTablet for tablet c4984a6b03214c4aae5786edbd574954 (DEFAULT_TABLE table=test-table [id=bed59d9967f642b68268bb3a09f88359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:09.022977 32162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4984a6b03214c4aae5786edbd574954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:09.024593 32080 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap starting.
I20260810 23:01:09.024971 32080 tablet_bootstrap.cc:654] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:09.026001 32080 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: No bootstrap required, opened a new log
I20260810 23:01:09.026058 32080 ts_tablet_manager.cc:1397] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.026211 32080 raft_consensus.cc:348] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.026284 32080 raft_consensus.cc:374] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:09.026336 32080 raft_consensus.cc:729] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940ae1cefc6044db9703c37ed3f6db82, State: Initialized, Role: FOLLOWER
I20260810 23:01:09.026345 32286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.022269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40611 (local address 127.0.0.1:35040)
0810 23:01:09.022384 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.022395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.022425 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.022768 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:09.022770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:09.022772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:09.022817 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.022819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.024040 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.024041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.025080 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.025083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.026070 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.026071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.026150 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.026171 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 23:01:09.026232 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 23:01:09.026277 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 23:01:09.026361 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.022410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.6.47.130:40611)
0810 23:01:09.022603 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.022608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.022619 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:01:09.022664 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.022666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.022680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.022752 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:09.022907 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.023935 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.025184 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.025982 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.026071 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.026132 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.026249 (+ 117us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.026287 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260810 23:01:09.026378 32080 consensus_queue.cc:260] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.026464 32080 ts_tablet_manager.cc:1428] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:09.027132 32418 tablet_service.cc:1467] Processing CreateTablet for tablet c4984a6b03214c4aae5786edbd574954 (DEFAULT_TABLE table=test-table [id=bed59d9967f642b68268bb3a09f88359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:09.027439 32418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4984a6b03214c4aae5786edbd574954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:09.028939 340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.022269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44171 (local address 127.0.0.1:44438)
0810 23:01:09.024720 (+ 2451us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.024733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.024792 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.025067 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:09.025068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:09.025070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:09.025132 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.025134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.027107 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.027108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.027949 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.027952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.028687 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.028688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.028763 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.028767 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:09.028825 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:01:09.028867 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2436,"mutex_wait_us":21,"thread_start_us":55,"threads_started":1}
I20260810 23:01:09.028954 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.022475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.6.47.131:44171)
0810 23:01:09.022686 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.022690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.024858 (+ 2168us) server_negotiation.cc:407] Connection header received
0810 23:01:09.024906 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.024909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.024927 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.025001 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:09.025359 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.026970 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.028042 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.028586 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.028675 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.028734 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.028845 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.028882 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
I20260810 23:01:09.029958 32666 tablet_service.cc:1467] Processing CreateTablet for tablet c4984a6b03214c4aae5786edbd574954 (DEFAULT_TABLE table=test-table [id=bed59d9967f642b68268bb3a09f88359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:09.030324 32666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4984a6b03214c4aae5786edbd574954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:09.030736 345 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Bootstrap starting.
I20260810 23:01:09.031440 345 tablet_bootstrap.cc:654] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:09.032034 345 log.cc:824] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:09.032888 345 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: No bootstrap required, opened a new log
I20260810 23:01:09.032945 345 ts_tablet_manager.cc:1397] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:01:09.033581 347 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Bootstrap starting.
I20260810 23:01:09.034276 347 tablet_bootstrap.cc:654] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:09.034605 345 raft_consensus.cc:348] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.034765 345 raft_consensus.cc:374] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:09.034763 347 log.cc:824] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:09.034816 345 raft_consensus.cc:729] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc4ed5748b3497ea095b35076446748, State: Initialized, Role: FOLLOWER
I20260810 23:01:09.034960 345 consensus_queue.cc:260] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.035302 345 ts_tablet_manager.cc:1428] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:01:09.035305 32540 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
I20260810 23:01:09.035565 347 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: No bootstrap required, opened a new log
I20260810 23:01:09.035651 347 ts_tablet_manager.cc:1397] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:01:09.037200 347 raft_consensus.cc:348] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.037403 347 raft_consensus.cc:374] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:09.037460 347 raft_consensus.cc:729] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f07487235246218c0b9b9254494013, State: Initialized, Role: FOLLOWER
I20260810 23:01:09.037590 347 consensus_queue.cc:260] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.037899 347 ts_tablet_manager.cc:1428] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:01:09.037899 331 heartbeater.cc:503] Master 127.6.47.190:44331 was elected leader, sending a full tablet report...
W20260810 23:01:09.113615 32541 tablet.cc:2369] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:01:09.209067 32276 tablet.cc:2369] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:01:09.212563 353 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:09.212675 353 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.213011 353 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:09.215883 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.213292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:55335 (local address 127.6.47.129:37015)
0810 23:01:09.213351 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.213353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.213362 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:01:09.213398 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.213399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.213417 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.213513 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:09.213700 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.214219 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215143 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.215535 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215627 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.215700 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.215785 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.215825 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:01:09.215883 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.213205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.6.47.130:55335)
0810 23:01:09.213248 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.213262 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.213303 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.213521 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:09.213523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:09.213525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:09.213620 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.213621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.214315 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.214316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.215040 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.215042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.215630 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.215708 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.215712 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:09.215763 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:01:09.215807 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 23:01:09.215996 343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.213291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:51359 (local address 127.6.47.131:44171)
0810 23:01:09.213367 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.213369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.213613 (+ 244us) server_negotiation.cc:407] Connection header received
0810 23:01:09.213694 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.213696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.213708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.213754 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:09.213926 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.214372 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215264 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.215630 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215731 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.215740 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.215899 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.215931 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 23:01:09.216004 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.213205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44171 (local address 127.6.47.130:51359)
0810 23:01:09.213527 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.213537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.213620 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.213762 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:09.213763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:09.213764 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:09.213854 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.213855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.214460 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.214462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.215160 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:09.215162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.215734 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.215736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:09.215810 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.215820 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 23:01:09.215880 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:01:09.215921 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":149,"thread_start_us":123,"threads_started":1}
I20260810 23:01:09.216246 32196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4984a6b03214c4aae5786edbd574954" candidate_uuid: "7fc4ed5748b3497ea095b35076446748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940ae1cefc6044db9703c37ed3f6db82" is_pre_election: true
I20260810 23:01:09.216346 32700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4984a6b03214c4aae5786edbd574954" candidate_uuid: "7fc4ed5748b3497ea095b35076446748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f07487235246218c0b9b9254494013" is_pre_election: true
I20260810 23:01:09.216437 32196 raft_consensus.cc:2454] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fc4ed5748b3497ea095b35076446748 in term 0.
I20260810 23:01:09.216517 32700 raft_consensus.cc:2454] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fc4ed5748b3497ea095b35076446748 in term 0.
I20260810 23:01:09.216724 32319 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fc4ed5748b3497ea095b35076446748, 940ae1cefc6044db9703c37ed3f6db82; no voters:
I20260810 23:01:09.216894 353 raft_consensus.cc:2781] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:09.216956 353 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:09.216997 353 raft_consensus.cc:3037] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:09.217803 353 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.217952 353 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [CANDIDATE]: Term 1 election: Requested vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015)
I20260810 23:01:09.218135 32700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4984a6b03214c4aae5786edbd574954" candidate_uuid: "7fc4ed5748b3497ea095b35076446748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f07487235246218c0b9b9254494013"
I20260810 23:01:09.218115 32196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4984a6b03214c4aae5786edbd574954" candidate_uuid: "7fc4ed5748b3497ea095b35076446748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940ae1cefc6044db9703c37ed3f6db82"
I20260810 23:01:09.218215 32700 raft_consensus.cc:3037] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:09.218225 32196 raft_consensus.cc:3037] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:09.219113 32196 raft_consensus.cc:2454] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fc4ed5748b3497ea095b35076446748 in term 1.
I20260810 23:01:09.219156 32700 raft_consensus.cc:2454] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fc4ed5748b3497ea095b35076446748 in term 1.
I20260810 23:01:09.219316 32321 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013; no voters:
I20260810 23:01:09.219417 353 raft_consensus.cc:2781] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:09.219588 353 raft_consensus.cc:686] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 1 LEADER]: Becoming Leader. State: Replica: 7fc4ed5748b3497ea095b35076446748, State: Running, Role: LEADER
I20260810 23:01:09.219725 353 consensus_queue.cc:237] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.220705 31929 catalog_manager.cc:5495] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fc4ed5748b3497ea095b35076446748 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "7fc4ed5748b3497ea095b35076446748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } health_report { overall_health: UNKNOWN } } }
W20260810 23:01:09.249778 332 tablet.cc:2369] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:01:09.256626 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.254250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.6.47.130:40611)
0810 23:01:09.254318 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:09.254320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:09.254328 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:01:09.254366 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:09.254367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:09.254386 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:09.254461 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:09.254598 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.255058 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.255917 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:09.256092 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:09.256141 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:09.256285 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:09.256393 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:09.256452 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:09.256525 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 23:01:09.256550 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:01:09.258656 32700 raft_consensus.cc:1261] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Refusing update from remote peer 7fc4ed5748b3497ea095b35076446748: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:09.258663 32196 raft_consensus.cc:1261] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Refusing update from remote peer 7fc4ed5748b3497ea095b35076446748: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:09.259115 353 consensus_queue.cc:1035] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:09.259317 360 consensus_queue.cc:1035] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:09.261833 367 mvcc.cc:205] Tried to move back new op lower bound from 7317106152480202752 to 7317106152324780032. Current Snapshot: MvccSnapshot[applied={T|T < 7317106152480202752}]
I20260810 23:01:09.264451 31928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:09.264595 31928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:09.265338 31928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Sending DeleteTablet for 3 replicas of tablet c4984a6b03214c4aae5786edbd574954
I20260810 23:01:09.265671 32418 tablet_service.cc:1514] Processing DeleteTablet for tablet c4984a6b03214c4aae5786edbd574954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:35040
I20260810 23:01:09.265676 32666 tablet_service.cc:1514] Processing DeleteTablet for tablet c4984a6b03214c4aae5786edbd574954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:44438
I20260810 23:01:09.265867 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 31993
I20260810 23:01:09.265890 32162 tablet_service.cc:1514] Processing DeleteTablet for tablet c4984a6b03214c4aae5786edbd574954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:09 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:42244
I20260810 23:01:09.265936 384 tablet_replica.cc:330] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: stopping tablet replica
I20260810 23:01:09.266036 384 raft_consensus.cc:2229] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:09.266129 384 raft_consensus.cc:2258] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:09.266366 385 tablet_replica.cc:330] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: stopping tablet replica
I20260810 23:01:09.266443 385 raft_consensus.cc:2229] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:01:09.266520 385 raft_consensus.cc:2258] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:09.275189 384 ts_tablet_manager.cc:1905] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:09.277047 385 ts_tablet_manager.cc:1905] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:09.277302 384 ts_tablet_manager.cc:1918] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:09.277441 384 log.cc:1196] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/c4984a6b03214c4aae5786edbd574954
I20260810 23:01:09.277854 384 ts_tablet_manager.cc:1939] T c4984a6b03214c4aae5786edbd574954 P 94f07487235246218c0b9b9254494013: Deleting consensus metadata
I20260810 23:01:09.278457 31901 catalog_manager.cc:4841] TS 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): tablet c4984a6b03214c4aae5786edbd574954 (table test-table [id=bed59d9967f642b68268bb3a09f88359]) successfully deleted
I20260810 23:01:09.279150 385 ts_tablet_manager.cc:1918] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:09.279271 385 log.cc:1196] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/c4984a6b03214c4aae5786edbd574954
I20260810 23:01:09.279588 385 ts_tablet_manager.cc:1939] T c4984a6b03214c4aae5786edbd574954 P 7fc4ed5748b3497ea095b35076446748: Deleting consensus metadata
I20260810 23:01:09.280021 31900 catalog_manager.cc:4841] TS 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): tablet c4984a6b03214c4aae5786edbd574954 (table test-table [id=bed59d9967f642b68268bb3a09f88359]) successfully deleted
W20260810 23:01:09.282393 31900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:37015 (error 108)
I20260810 23:01:09.282661 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 32281
I20260810 23:01:09.285346 340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.282861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:37015 (local address 127.0.0.1:42246)
0810 23:01:09.285244 (+ 2383us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.285294 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2359}
W20260810 23:01:09.285521 31900 catalog_manager.cc:4584] TS 940ae1cefc6044db9703c37ed3f6db82 (127.6.47.129:37015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4984a6b03214c4aae5786edbd574954: Network error: Client connection negotiation failed: client connection to 127.6.47.129:37015: connect: Connection refused (error 111)
I20260810 23:01:09.294842 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 32546
I20260810 23:01:09.308467 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 31873
I20260810 23:01:09.319883 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--webserver_interface=127.6.47.190
--webserver_port=35807
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:44331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:09.404256 389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:09.404486 389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:09.404544 389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:09.405755 389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:09.405825 389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:09.405853 389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:09.405885 389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:09.405910 389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:09.407392 389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:44331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=35807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:09.407666 389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:09.407895 389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:09.412791 402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.412791 405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.412839 400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:09.412876 389 server_base.cc:1034] running on GCE node
I20260810 23:01:09.413229 389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:09.413534 389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:09.414705 389 hybrid_clock.cc:648] HybridClock initialized: now 1786402869414703 us; error 41 us; skew 500 ppm
Aug 10 23:01:09 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:09.417645 389 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:09.418061 389 webserver.cc:460] Webserver started at http://127.6.47.190:35807/ using document root <none> and password file <none>
I20260810 23:01:09.418319 389 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:09.418407 389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:09.419651 389 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.420329 420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.420554 389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.420663 389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2e64477a3fa741128f318bdcfa848260"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "8ee39e17f7ce6ce882115421cae62c6c"
server_key_iv: "487f32d1d90e9486c79746c5ab2610de"
server_key_version: "encryptionkey@0"
I20260810 23:01:09.421008 389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:09.434597 389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:09.436390 389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:09.436614 389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:09.440688 389 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:44331
I20260810 23:01:09.440727 506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:44331 every 8 connection(s)
I20260810 23:01:09.441111 389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:09.442688 509 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:01:09.443351 509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap starting.
I20260810 23:01:09.445621 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 389
I20260810 23:01:09.445955 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:37015
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=43947
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:09.453519 509 log.cc:824] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:09.455866 509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:09.456187 509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260: Bootstrap complete.
I20260810 23:01:09.458581 509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:09.458817 509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Initialized, Role: FOLLOWER
I20260810 23:01:09.458987 509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:09.459062 509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:09.459128 509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:09.459180 509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:01:09.460162 509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:09.460280 509 leader_election.cc:302] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e64477a3fa741128f318bdcfa848260; no voters:
I20260810 23:01:09.460445 509 leader_election.cc:288] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 23:01:09.460588 522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 23:01:09.460709 522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [term 3 LEADER]: Becoming Leader. State: Replica: 2e64477a3fa741128f318bdcfa848260, State: Running, Role: LEADER
I20260810 23:01:09.460757 509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:09.460806 522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } }
I20260810 23:01:09.461102 522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e64477a3fa741128f318bdcfa848260. Latest consensus state: current_term: 3 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:09.461218 522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:09.461105 523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2e64477a3fa741128f318bdcfa848260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e64477a3fa741128f318bdcfa848260" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44331 } } }
I20260810 23:01:09.461314 523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e64477a3fa741128f318bdcfa848260 [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:09.462388 532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:09.463212 532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:09.463343 532 catalog_manager.cc:1259] Loaded cluster ID: 67a9eb0d44ed414ca347881e79eb57a5
I20260810 23:01:09.463384 532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 23:01:09.463923 532 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7286bf49ad google::LogMessage::Fail() at ??:0
@ 0x7f7286bf788a google::LogMessage::SendToLog() at ??:0
@ 0x7f7286bf44de google::LogMessage::Flush() at ??:0
@ 0x7f7286bf5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f728eb150f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f728eb1e8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7288ac20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7288ab7954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f728991b6db start_thread at ??:0
@ 0x7f72856f261f clone at ??:0
W20260810 23:01:09.556309 516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:09.556607 516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:09.556638 516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:09.557847 516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:09.557891 516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:09.557919 516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:01:09.559458 516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:37015
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=43947
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:09.559743 516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:09.559998 516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:09.565408 580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.565434 588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.565481 582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:09.565717 516 server_base.cc:1034] running on GCE node
I20260810 23:01:09.565961 516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:09.566229 516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:09.567396 516 hybrid_clock.cc:648] HybridClock initialized: now 1786402869567370 us; error 45 us; skew 500 ppm
Aug 10 23:01:09 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:09.570348 516 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:01:09.570948 516 webserver.cc:460] Webserver started at http://127.6.47.129:43947/ using document root <none> and password file <none>
I20260810 23:01:09.571206 516 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:09.571280 516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:09.572759 516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:01:09.573532 597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.573734 516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.573858 516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "940ae1cefc6044db9703c37ed3f6db82"
format_stamp: "Formatted at 2026-08-10 23:01:07 on dist-test-slave-rwrg"
server_key: "9ebd8e38a84e72f5d8bb77a3b2c20b79"
server_key_iv: "0f0fb8bcfefbf9137e28a07a901f0fc5"
server_key_version: "encryptionkey@0"
I20260810 23:01:09.574312 516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:09.587842 516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:09.590361 516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:09.590525 516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:09.590762 516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:09.591281 610 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:01:09.592973 516 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:01:09.593035 516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:01:09.593070 516 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:01:09.593979 516 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:01:09.594036 516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:01:09.594099 610 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap starting.
I20260810 23:01:09.601704 516 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:37015
I20260810 23:01:09.601903 814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:37015 every 8 connection(s)
I20260810 23:01:09.602171 516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:01:09.602571 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 516
I20260810 23:01:09.602895 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:40611
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=44991
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:09.605252 610 log.cc:824] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:09.609442 610 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:09.609760 610 tablet_bootstrap.cc:492] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Bootstrap complete.
I20260810 23:01:09.611043 610 ts_tablet_manager.cc:1397] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.010s
I20260810 23:01:09.612469 610 raft_consensus.cc:348] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.612830 610 raft_consensus.cc:729] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940ae1cefc6044db9703c37ed3f6db82, State: Initialized, Role: FOLLOWER
I20260810 23:01:09.612986 610 consensus_queue.cc:260] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:09.613308 610 ts_tablet_manager.cc:1428] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
W20260810 23:01:09.683179 821 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:01:09.605199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:43415)
0810 23:01:09.605478 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.605530 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:09.606177 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:09.682962 (+ 76785us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":111,"threads_started":1}
W20260810 23:01:09.683415 817 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:44331 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:44331: Client connection negotiation failed: client connection to 127.6.47.190:44331: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:01:09.683768 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.683647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:41583)
0810 23:01:09.683701 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.683747 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:01:09.684158 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.684066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:46107)
0810 23:01:09.684118 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.684143 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 23:01:09.684293 817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:01:09.696121 820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:09.696313 820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:09.696336 820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:09.697479 820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:09.697553 820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:09.697577 820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:01:09.698937 820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:40611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=44991
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:09.699131 820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:09.699304 820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:09.704306 832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.704258 830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:09.704257 820 server_base.cc:1034] running on GCE node
W20260810 23:01:09.704276 844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:09.704692 820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:09.704994 820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:09.706182 820 hybrid_clock.cc:648] HybridClock initialized: now 1786402869706160 us; error 47 us; skew 500 ppm
Aug 10 23:01:09 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:09.709847 820 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:01:09.710397 820 webserver.cc:460] Webserver started at http://127.6.47.130:44991/ using document root <none> and password file <none>
I20260810 23:01:09.710664 820 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:09.710714 820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:09.712675 820 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260810 23:01:09.713992 853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.714231 820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.714331 820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "7fc4ed5748b3497ea095b35076446748"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "100df16964ac4315c3e536a4f882b429"
server_key_iv: "67ffe3d8b1f10c01be4802d4c1771b8d"
server_key_version: "encryptionkey@0"
I20260810 23:01:09.714692 820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:09.742198 820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:09.744688 820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:09.744843 820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:09.745103 820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:09.745441 820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:09.745481 820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.745502 820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:09.745548 820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.751499 820 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:40611
I20260810 23:01:09.751583 1062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:40611 every 8 connection(s)
I20260810 23:01:09.751940 820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:01:09.753073 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.752690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:44497)
0810 23:01:09.752865 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.752936 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260810 23:01:09.753500 1065 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:44331 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:44331: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
I20260810 23:01:09.753793 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.753705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:32943)
0810 23:01:09.753750 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.753776 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 23:01:09.754027 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.753981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:47359)
0810 23:01:09.753999 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.754016 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260810 23:01:09.754149 1065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:01:09.761401 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 820
I20260810 23:01:09.761684 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:44171
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:44331
--webserver_port=32783
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:46399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:09.845798 1069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:09.846016 1069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:09.846040 1069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:09.847119 1069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:09.847159 1069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:09.847185 1069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:01:09.848587 1069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:46399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:44171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=32783
--tserver_master_addrs=127.6.47.190:44331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:09.848764 1069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:09.848937 1069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:09.854187 1076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.854194 1084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:09.854194 1078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:09.854456 1069 server_base.cc:1034] running on GCE node
I20260810 23:01:09.854789 1069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:09.855026 1069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:09.856189 1069 hybrid_clock.cc:648] HybridClock initialized: now 1786402869856176 us; error 32 us; skew 500 ppm
Aug 10 23:01:09 dist-test-slave-rwrg krb5kdc[30896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:09.858755 1069 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:01:09.859233 1069 webserver.cc:460] Webserver started at http://127.6.47.131:32783/ using document root <none> and password file <none>
I20260810 23:01:09.859467 1069 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:09.859544 1069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:09.860898 1069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:09.861532 1095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.861711 1069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:01:09.861822 1069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "94f07487235246218c0b9b9254494013"
format_stamp: "Formatted at 2026-08-10 23:01:08 on dist-test-slave-rwrg"
server_key: "c451d1b35129260cd0c5b0603946f608"
server_key_iv: "2dd84f8944697a49dc58f265c05649a6"
server_key_version: "encryptionkey@0"
I20260810 23:01:09.862164 1069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:09.877140 1069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:09.879150 1069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:09.879324 1069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:09.879567 1069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:09.879916 1069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:09.879969 1069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.880002 1069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:09.880028 1069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:09.886415 1069 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:44171
I20260810 23:01:09.886468 1290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:44171 every 8 connection(s)
I20260810 23:01:09.886801 1069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:01:09.887852 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.887470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:52365)
0810 23:01:09.887645 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.887716 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260810 23:01:09.888265 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 1069
W20260810 23:01:09.888465 1292 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:44331 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:44331: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
I20260810 23:01:09.888832 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.888722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:50101)
0810 23:01:09.888774 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.888802 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:01:09.889119 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:09.889049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:39223)
0810 23:01:09.889077 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:09.889099 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260810 23:01:09.889258 1292 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:01:10.685058 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:10.684795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:52571)
0810 23:01:10.684951 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:10.684992 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260810 23:01:10.754733 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:10.754545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:43599)
0810 23:01:10.754660 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:10.754691 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260810 23:01:10.889739 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:10.889533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:49293)
0810 23:01:10.889647 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:10.889686 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260810 23:01:11.178354 1302 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:11.178524 1302 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:11.178934 1302 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
Aug 10 23:01:11 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.130@KRBTEST.COM
Aug 10 23:01:11 dist-test-slave-rwrg krb5kdc[30896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.131@KRBTEST.COM
I20260810 23:01:11.192546 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.179160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:40611 (local address 127.6.47.129:36949)
0810 23:01:11.179297 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:11.179308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:11.179378 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:11.180214 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:11.180215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:11.180653 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:11.180841 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:11.180847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.181910 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.181911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:11.182868 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:11.182871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.183448 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.183449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:11.183555 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:11.187080 (+ 3525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:11.187088 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:11.188772 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:11.190791 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:11.190799 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:11.190801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:11.191089 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:11.191327 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:11.191329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:11.191468 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:11.191935 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:11.191941 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:11.192104 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 23:01:11.192223 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 23:01:11.192386 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":17,"thread_start_us":74,"threads_started":1}
I20260810 23:01:11.192561 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.179298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:36949 (local address 127.6.47.130:40611)
0810 23:01:11.179488 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:11.179497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:11.179528 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:01:11.179941 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:11.179943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:11.179982 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:11.180129 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:11.180961 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.181802 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.182966 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.183357 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.183473 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:11.188897 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:11.188902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:11.188902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:11.188916 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:11.190704 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191174 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:11.191176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:11.191176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:11.191257 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191544 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:11.191544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:11.191545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:11.191857 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:11.191941 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:11.192273 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 23:01:11.192402 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260810 23:01:11.192735 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.179126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44171 (local address 127.6.47.129:38823)
0810 23:01:11.179163 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:11.179175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:11.179240 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:11.180162 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:11.180173 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:11.180647 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:11.180841 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:11.180847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.181935 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.181936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:11.182850 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:11.182854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.183449 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.183450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:11.183554 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:11.187102 (+ 3548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:11.187106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:11.189202 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:11.191427 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:11.191428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:11.191637 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:11.191867 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:11.191868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:11.191937 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:11.192374 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:11.192375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:11.192420 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 23:01:11.192527 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 23:01:11.192629 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":70}
I20260810 23:01:11.192878 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.179275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:38823 (local address 127.6.47.131:44171)
0810 23:01:11.179450 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:11.179463 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:11.179504 (+ 41us) server_negotiation.cc:407] Connection header received
0810 23:01:11.179941 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:11.179943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:11.179982 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:11.180116 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:11.180971 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.181813 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.182949 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:11.183357 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:11.183473 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:11.189318 (+ 5845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:11.189323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:11.189323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:11.189336 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:11.191351 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:11.191718 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:11.191721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:11.191721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:11.191799 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:11.192005 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:11.192005 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:11.192006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:11.192299 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:11.192380 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:11.192590 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 23:01:11.192743 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260810 23:01:11.193783 602 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
W20260810 23:01:11.194023 599 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
I20260810 23:01:11.194088 599 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82; no voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013
I20260810 23:01:11.194217 1302 raft_consensus.cc:2726] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:01:11.685644 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.685501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:56599)
0810 23:01:11.685563 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:11.685602 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:01:11.755380 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.755188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:37713)
0810 23:01:11.755306 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:11.755344 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260810 23:01:11.890343 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:11.890140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:42927)
0810 23:01:11.890268 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:11.890305 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260810 23:01:12.686305 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:12.686090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:34957)
0810 23:01:12.686221 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:12.686255 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260810 23:01:12.756007 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:12.755818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:39615)
0810 23:01:12.755936 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:12.755970 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260810 23:01:12.891036 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:12.890833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:49339)
0810 23:01:12.890955 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:12.890994 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260810 23:01:13.170675 1319 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:13.170778 1319 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:13.170939 1319 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
W20260810 23:01:13.171357 599 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
W20260810 23:01:13.171437 602 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
I20260810 23:01:13.171530 602 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82; no voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013
I20260810 23:01:13.171626 1319 raft_consensus.cc:2726] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:01:13.687039 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:13.686823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:57311)
0810 23:01:13.686954 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:13.686993 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260810 23:01:13.756739 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:13.756556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:55919)
0810 23:01:13.756673 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:13.756701 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260810 23:01:13.891755 1327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:13.891547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:45215)
0810 23:01:13.891674 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:13.891713 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260810 23:01:14.687788 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:14.687565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:54511)
0810 23:01:14.687703 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:14.687741 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260810 23:01:14.757393 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:14.757190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:58983)
0810 23:01:14.757309 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:14.757343 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260810 23:01:14.892450 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:14.892254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:36859)
0810 23:01:14.892378 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:14.892413 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260810 23:01:15.688651 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:15.688360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:59261)
0810 23:01:15.688549 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:15.688602 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260810 23:01:15.758188 1340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:15.757979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:56085)
0810 23:01:15.758106 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:15.758145 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260810 23:01:15.893146 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:15.892921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:54655)
0810 23:01:15.893058 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:15.893090 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260810 23:01:16.047624 1347 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:16.047722 1347 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:16.047892 1347 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
W20260810 23:01:16.048352 599 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
W20260810 23:01:16.048442 602 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
I20260810 23:01:16.048534 602 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82; no voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013
I20260810 23:01:16.048628 1347 raft_consensus.cc:2726] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:01:16.689347 1348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:16.689098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:46623)
0810 23:01:16.689264 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:16.689298 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":87,"threads_started":1}
I20260810 23:01:16.758848 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:16.758634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:40871)
0810 23:01:16.758776 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:16.758806 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260810 23:01:16.893834 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:16.893564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:50347)
0810 23:01:16.893723 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:16.893777 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260810 23:01:17.690173 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:17.689953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:59719)
0810 23:01:17.690083 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:17.690125 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260810 23:01:17.759491 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:17.759281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:53183)
0810 23:01:17.759417 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:17.759453 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260810 23:01:17.894495 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:17.894287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:37359)
0810 23:01:17.894420 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:17.894455 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260810 23:01:18.690873 1362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:18.690651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:48457)
0810 23:01:18.690792 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:18.690825 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260810 23:01:18.760429 1363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:18.760182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:60649)
0810 23:01:18.760343 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:18.760383 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260810 23:01:18.895366 1366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:18.895128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:42223)
0810 23:01:18.895280 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:18.895320 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260810 23:01:19.247756 1367 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:19.247875 1367 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:19.248104 1367 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
W20260810 23:01:19.248575 599 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
W20260810 23:01:19.248646 602 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
I20260810 23:01:19.248675 599 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82; no voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013
I20260810 23:01:19.248744 1367 raft_consensus.cc:2726] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:01:19.691610 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:19.691378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:45919)
0810 23:01:19.691521 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:19.691563 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260810 23:01:19.761122 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:19.760885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:46105)
0810 23:01:19.761042 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:19.761080 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260810 23:01:19.896054 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:19.895810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:56067)
0810 23:01:19.895954 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:19.895991 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260810 23:01:20.692413 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:20.692190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:59987)
0810 23:01:20.692327 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:20.692366 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260810 23:01:20.761795 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:20.761604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:48965)
0810 23:01:20.761723 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:20.761758 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260810 23:01:20.896829 1381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:20.896545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:60197)
0810 23:01:20.896746 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:20.896786 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
I20260810 23:01:21.693207 1382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:21.692941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:43419)
0810 23:01:21.693089 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:21.693139 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260810 23:01:21.762568 1383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:21.762309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:45517)
0810 23:01:21.762430 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:21.762508 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260810 23:01:21.897483 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:21.897265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:34175)
0810 23:01:21.897406 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:21.897440 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260810 23:01:22.693828 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:22.693607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:59509)
0810 23:01:22.693731 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:22.693768 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260810 23:01:22.763203 1389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:22.763008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:55939)
0810 23:01:22.763132 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:22.763164 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260810 23:01:22.898164 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:22.897959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:48273)
0810 23:01:22.898080 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:22.898120 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260810 23:01:23.694528 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:23.694297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:44253)
0810 23:01:23.694443 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:23.694479 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260810 23:01:23.763826 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:23.763607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:51707)
0810 23:01:23.763749 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:23.763785 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260810 23:01:23.898944 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:23.898712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:41805)
0810 23:01:23.898865 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:23.898897 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260810 23:01:24.070896 1396 raft_consensus.cc:482] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:24.070990 1396 raft_consensus.cc:504] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f07487235246218c0b9b9254494013" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44171 } } peers { permanent_uuid: "7fc4ed5748b3497ea095b35076446748" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 40611 } } peers { permanent_uuid: "940ae1cefc6044db9703c37ed3f6db82" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 37015 } }
I20260810 23:01:24.071166 1396 leader_election.cc:288] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 94f07487235246218c0b9b9254494013 (127.6.47.131:44171), 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611)
W20260810 23:01:24.071568 602 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 94f07487235246218c0b9b9254494013 (127.6.47.131:44171): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
W20260810 23:01:24.071651 599 leader_election.cc:341] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7fc4ed5748b3497ea095b35076446748 (127.6.47.130:40611): Not found: Tablet not found: c4984a6b03214c4aae5786edbd574954
I20260810 23:01:24.071748 599 leader_election.cc:302] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 940ae1cefc6044db9703c37ed3f6db82; no voters: 7fc4ed5748b3497ea095b35076446748, 94f07487235246218c0b9b9254494013
I20260810 23:01:24.071820 1396 raft_consensus.cc:2726] T c4984a6b03214c4aae5786edbd574954 P 940ae1cefc6044db9703c37ed3f6db82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:01:24.695396 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:24.695086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.129:42259)
0810 23:01:24.695304 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:24.695346 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":68,"threads_started":1}
I20260810 23:01:24.764562 1401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:24.764304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.130:44389)
0810 23:01:24.764428 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:24.764465 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260810 23:01:24.889467 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 516
I20260810 23:01:24.899724 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:24.899473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:44331 (local address 127.6.47.131:39413)
0810 23:01:24.899635 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:24.899669 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:44331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260810 23:01:24.903290 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 820
I20260810 23:01:24.916128 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 1069
2026-08-10T23:01:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18698 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:01:24 dist-test-slave-rwrg krb5kdc[1408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:01:24 dist-test-slave-rwrg krb5kdc[1408](info): set up 2 sockets
Aug 10 23:01:24 dist-test-slave-rwrg krb5kdc[1408](info): commencing operation
krb5kdc: starting...
W20260810 23:01:26.220558 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:01:26.341475 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:26.424057 1438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:26.424302 1438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:26.424342 1438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:26.425482 1438 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:01:26.425546 1438 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:01:26.425568 1438 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:01:26.425588 1438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:26.425617 1438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:26.425649 1438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:26.425673 1438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:26.425702 1438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:26.427160 1438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:26.427459 1438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:26.427690 1438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:26.433231 1453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.433192 1451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.433285 1438 server_base.cc:1034] running on GCE node
W20260810 23:01:26.433157 1456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.433647 1438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:26.433970 1438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:26.435184 1438 hybrid_clock.cc:648] HybridClock initialized: now 1786402886435164 us; error 33 us; skew 500 ppm
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:26.437813 1438 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:26.438277 1438 webserver.cc:460] Webserver started at http://127.6.47.190:37867/ using document root <none> and password file <none>
I20260810 23:01:26.438511 1438 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:26.438581 1438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:26.438740 1438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:26.439791 1438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "d35d8be73ac84a99b49e4d091c8a6baa"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "f7cda2f0bdb31633955204d3388fac42"
server_key_iv: "09d322a312636eb99a0fbdf9d38aa737"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.440187 1438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "d35d8be73ac84a99b49e4d091c8a6baa"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "f7cda2f0bdb31633955204d3388fac42"
server_key_iv: "09d322a312636eb99a0fbdf9d38aa737"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.441670 1438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:01:26.442484 1466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.442703 1438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.442807 1438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "d35d8be73ac84a99b49e4d091c8a6baa"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "f7cda2f0bdb31633955204d3388fac42"
server_key_iv: "09d322a312636eb99a0fbdf9d38aa737"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.442894 1438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:26.460644 1438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:26.463043 1438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:26.463196 1438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:26.467510 1438 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:38981
I20260810 23:01:26.467579 1571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:38981 every 8 connection(s)
I20260810 23:01:26.467919 1438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:26.468828 1573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:26.471284 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap starting.
I20260810 23:01:26.471824 1573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:26.472095 1573 log.cc:824] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:26.472800 1573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: No bootstrap required, opened a new log
I20260810 23:01:26.474370 1573 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:26.474504 1573 raft_consensus.cc:374] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:26.474530 1573 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Initialized, Role: FOLLOWER
I20260810 23:01:26.474674 1573 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:26.474802 1573 raft_consensus.cc:388] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:26.474828 1573 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:26.474857 1573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:26.475368 1573 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:26.475453 1573 leader_election.cc:302] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d35d8be73ac84a99b49e4d091c8a6baa; no voters:
I20260810 23:01:26.475598 1573 leader_election.cc:288] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:01:26.475679 1578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:26.475839 1573 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:26.475852 1578 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 LEADER]: Becoming Leader. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Running, Role: LEADER
I20260810 23:01:26.476063 1578 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:26.476460 1579 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:01:26.476600 1579 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:26.476713 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 1438
I20260810 23:01:26.476816 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:01:26.476879 1592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:26.477046 6334 external_mini_cluster.cc:1428] Setting key dde788da97993c19bf782ef912a58668
I20260810 23:01:26.477725 1592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:26.476462 1584 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35d8be73ac84a99b49e4d091c8a6baa. Latest consensus state: current_term: 1 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:01:26.478161 1584 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:26.480612 1592 catalog_manager.cc:1347] Generated new cluster ID: 6fac4ba7eacd496c93bd8ec9bfc7ec76
I20260810 23:01:26.480671 1592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:26.487497 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.477754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.6.47.190:38981)
0810 23:01:26.478026 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:26.478036 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:26.478101 (+ 65us) server_negotiation.cc:407] Connection header received
0810 23:01:26.478482 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:26.478492 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:26.478555 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:26.478717 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:26.479519 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.480420 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.481257 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.481698 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.481817 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.484191 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:26.484194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:26.484195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:26.484206 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:26.485889 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.486287 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.486290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.486291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.486412 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.486639 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.486640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.486640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.486876 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:26.487022 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:26.487146 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 23:01:26.487289 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":97,"threads_started":1}
I20260810 23:01:26.498317 1592 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:01:26.498894 1592 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:01:26.504254 1592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Generated new TSK 0
I20260810 23:01:26.504423 1592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:01:26.540697 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:26.624190 1626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:26.624396 1626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:26.624418 1626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:26.625645 1626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:26.625690 1626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:26.625716 1626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:01:26.627140 1626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:26.627408 1626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:26.627647 1626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:26.632858 1638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.632912 1626 server_base.cc:1034] running on GCE node
W20260810 23:01:26.632915 1639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.632877 1641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.633291 1626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:26.633555 1626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:26.634735 1626 hybrid_clock.cc:648] HybridClock initialized: now 1786402886634717 us; error 35 us; skew 500 ppm
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:26.637315 1626 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:01:26.637843 1626 webserver.cc:460] Webserver started at http://127.6.47.129:33175/ using document root <none> and password file <none>
I20260810 23:01:26.638105 1626 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:26.638188 1626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:26.638350 1626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:26.639393 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "838b82c51197e15772a47f4121c3dfc1"
server_key_iv: "3f5a399436dfc0496b3f02baacfb4057"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.639787 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "838b82c51197e15772a47f4121c3dfc1"
server_key_iv: "3f5a399436dfc0496b3f02baacfb4057"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.641265 1626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:01:26.642045 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.642216 1626 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:01:26.642328 1626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "838b82c51197e15772a47f4121c3dfc1"
server_key_iv: "3f5a399436dfc0496b3f02baacfb4057"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.642416 1626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:26.660936 1626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:26.662822 1626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:26.662994 1626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:26.663239 1626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:26.663554 1626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:26.663611 1626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.663653 1626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:26.663682 1626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.669854 1626 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40291
I20260810 23:01:26.669914 1878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40291 every 8 connection(s)
I20260810 23:01:26.670257 1626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:01:26.675922 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 1626
I20260810 23:01:26.676025 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:01:26.676254 6334 external_mini_cluster.cc:1428] Setting key a9a1a8ef3bbdcb7d588e556b0be9f5eb
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:26.680328 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.671089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:39373 (local address 127.6.47.190:38981)
0810 23:01:26.671157 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:26.671159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:26.671344 (+ 185us) server_negotiation.cc:407] Connection header received
0810 23:01:26.671806 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:26.671807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:26.671819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:26.671875 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:26.672781 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.673296 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.674291 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.674487 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.674540 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.677009 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:26.677010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:26.677011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:26.677020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:26.678806 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.679358 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.679359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.679359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.679413 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.679700 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.679701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.679701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.679801 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:26.679852 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:26.680193 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 23:01:26.680266 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:01:26.680636 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.671017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:39373)
0810 23:01:26.671200 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:26.671241 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:26.671721 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:26.671926 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:26.671929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:26.672513 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:26.672703 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.672708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.673395 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.673396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.674213 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.674216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.674588 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.674589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.674706 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.675169 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:26.675177 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:26.676772 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:26.678908 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:26.678917 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:26.678920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:26.679277 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:26.679482 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:26.679483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:26.679483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:26.679631 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:26.679866 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:26.679872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:26.680037 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 23:01:26.680163 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 23:01:26.680346 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
I20260810 23:01:26.681197 1879 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:26.681310 1879 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:26.681501 1879 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:26.682018 1496 ts_manager.cc:194] Registered new tserver with Master: f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291)
I20260810 23:01:26.682667 1496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:39373
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:01:26.703644 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:26.786972 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:26.787176 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:26.787200 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:26.788234 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:26.788266 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:26.788290 1886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:01:26.789796 1886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:26.789975 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:26.790153 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:26.795001 1900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.795017 1896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.795055 1897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.795325 1886 server_base.cc:1034] running on GCE node
I20260810 23:01:26.795527 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:26.795782 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:26.796943 1886 hybrid_clock.cc:648] HybridClock initialized: now 1786402886796927 us; error 26 us; skew 500 ppm
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:26.799618 1886 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:01:26.800081 1886 webserver.cc:460] Webserver started at http://127.6.47.130:34645/ using document root <none> and password file <none>
I20260810 23:01:26.800319 1886 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:26.800385 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:26.800562 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:26.801625 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "26b275414d5c580003a067dc7c3d2ed9"
server_key_iv: "954479d94d0705f734875b91596dcbfc"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.802009 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "26b275414d5c580003a067dc7c3d2ed9"
server_key_iv: "954479d94d0705f734875b91596dcbfc"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.803339 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:01:26.804078 1914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.804267 1886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:26.804378 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "26b275414d5c580003a067dc7c3d2ed9"
server_key_iv: "954479d94d0705f734875b91596dcbfc"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.804453 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:26.822448 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:26.824409 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:26.824622 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:26.824868 1886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:26.825209 1886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:26.825273 1886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.825316 1886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:26.825343 1886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.831723 1886 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:36083
I20260810 23:01:26.831804 2149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:36083 every 8 connection(s)
I20260810 23:01:26.832147 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:26.840302 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 1886
I20260810 23:01:26.840408 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:01:26.840626 6334 external_mini_cluster.cc:1428] Setting key 0c985f6b6776722a298a4df6561704f3
I20260810 23:01:26.841699 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.832937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:52473 (local address 127.6.47.190:38981)
0810 23:01:26.832997 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:26.832999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:26.833220 (+ 221us) server_negotiation.cc:407] Connection header received
0810 23:01:26.833778 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:26.833779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:26.833793 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:26.833858 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:26.834865 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.835383 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.836415 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.836702 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.836778 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.838997 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:26.838999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:26.838999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:26.839006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:26.840213 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.840714 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.840715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.840716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.840778 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:26.841038 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:26.841039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:26.841039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:26.841160 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:26.841248 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:26.841525 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 23:01:26.841611 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:01:26.841898 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.832909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:52473)
0810 23:01:26.833095 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:26.833133 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:26.833658 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:26.833932 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:26.833936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:26.834561 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:26.834763 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.834769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.835492 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.835493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.836303 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.836306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.836787 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.836789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.836916 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.837447 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:26.837455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:26.838902 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:26.840301 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:26.840310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:26.840312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:26.840615 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:26.840853 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:26.840854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:26.840854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:26.840968 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:26.841238 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:26.841246 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:26.841386 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:01:26.841492 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 23:01:26.841647 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260810 23:01:26.842404 2150 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:26.842501 2150 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:26.842700 2150 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:26.843103 1496 ts_manager.cc:194] Registered new tserver with Master: b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
I20260810 23:01:26.843621 1496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:52473
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:01:26.867936 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:26.951946 2158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:26.952137 2158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:26.952160 2158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:26.953302 2158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:26.953344 2158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:26.953372 2158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:01:26.954778 2158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:26.954952 2158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:26.955123 2158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:26.960072 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.960095 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:26.960103 2178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:26.960340 2158 server_base.cc:1034] running on GCE node
I20260810 23:01:26.960623 2158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:26.960861 2158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:26.962025 2158 hybrid_clock.cc:648] HybridClock initialized: now 1786402886961999 us; error 57 us; skew 500 ppm
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:26.964715 2158 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:01:26.965161 2158 webserver.cc:460] Webserver started at http://127.6.47.131:46073/ using document root <none> and password file <none>
I20260810 23:01:26.965382 2158 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:26.965422 2158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:26.965523 2158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:01:26.966455 2158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "ddbd718681476a1dee7defdb20fc7d17"
server_key_iv: "bc34fe48065a52eac2067634e33c15e9"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.966768 2158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "ddbd718681476a1dee7defdb20fc7d17"
server_key_iv: "bc34fe48065a52eac2067634e33c15e9"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.968003 2158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:01:26.968706 2190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.968861 2158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:26.968943 2158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "ddbd718681476a1dee7defdb20fc7d17"
server_key_iv: "bc34fe48065a52eac2067634e33c15e9"
server_key_version: "encryptionkey@0"
I20260810 23:01:26.968997 2158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:26.984184 2158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:26.986033 2158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:26.986142 2158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:26.986325 2158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:26.986634 2158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:26.986675 2158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.986727 2158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:26.986758 2158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:26.992674 2158 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:43877
I20260810 23:01:26.992748 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:43877 every 8 connection(s)
I20260810 23:01:26.993065 2158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:01:26.993358 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 2158
I20260810 23:01:26.993443 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:01:26.993624 6334 external_mini_cluster.cc:1428] Setting key f7975bacab6d4037c457c5f10ad6573d
Aug 10 23:01:26 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402886, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:27.002775 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.993809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:60767 (local address 127.6.47.190:38981)
0810 23:01:26.993867 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:26.993870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:26.994146 (+ 276us) server_negotiation.cc:407] Connection header received
0810 23:01:26.994811 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:26.994812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:26.994824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:26.994872 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:26.995869 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.996392 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.997444 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.997652 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.997742 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.999964 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:26.999966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:26.999966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:26.999974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:27.001329 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:27.001768 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:27.001770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:27.001770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:27.001838 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:27.002099 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:27.002100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:27.002101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:27.002232 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:27.002308 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.002594 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.002679 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:01:27.002985 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:26.993808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:60767)
0810 23:01:26.994029 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:26.994080 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:26.994704 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:26.994952 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:26.994956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:26.995575 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:26.995771 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.995776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.996515 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.996516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.997326 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:26.997328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:26.997756 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:26.997757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:26.997884 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:26.998408 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:26.998417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:26.999866 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:27.001419 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:27.001427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:27.001429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:27.001687 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:27.001919 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:27.001920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:27.001920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:27.002029 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:27.002310 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:27.002318 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:27.002456 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.002567 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.002740 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260810 23:01:27.003505 2414 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:27.003621 2414 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:27.003811 2414 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:27.004277 1496 ts_manager.cc:194] Registered new tserver with Master: 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877)
I20260810 23:01:27.004817 1496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:60767
I20260810 23:01:27.005419 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:01:27.012807 1615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.006802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33270 (local address 127.6.47.190:38981)
0810 23:01:27.006848 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.006850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.006920 (+ 70us) server_negotiation.cc:407] Connection header received
0810 23:01:27.007014 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.007031 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.007042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.007100 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:27.007711 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.008220 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.009030 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.009223 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.009338 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.010457 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:27.010458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:27.010459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:27.010466 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:27.011572 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:27.011943 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:27.011945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:27.011945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:27.012005 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:27.012233 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:27.012234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:27.012234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:27.012382 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:27.012494 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.012623 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.012708 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:01:27.014410 1495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:01:27.015983 1495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:01:27.022221 2440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:43877 (local address 127.0.0.1:36176)
0810 23:01:27.018615 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.018651 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:27.018737 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:27.018936 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:27.018940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:27.018942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:27.019020 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.019022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020101 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.020102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.021167 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.021170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.021872 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.021873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.021970 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.021988 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.022083 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.022145 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":9,"thread_start_us":80,"threads_started":1}
I20260810 23:01:27.022229 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.6.47.131:43877)
0810 23:01:27.018532 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.018537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.018748 (+ 211us) server_negotiation.cc:407] Connection header received
0810 23:01:27.018821 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.018824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.018840 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.018917 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:27.019117 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020000 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.021257 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.021781 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.021870 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.021931 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.022099 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.022154 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260810 23:01:27.022723 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:40291 (local address 127.0.0.1:44114)
0810 23:01:27.018700 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.018709 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:27.018768 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:27.019038 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:27.019039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:27.019041 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:27.019117 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.019119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020391 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.020392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.021256 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.021259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.022214 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.022215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.022539 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.022545 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.022600 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.022641 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":102,"threads_started":1}
I20260810 23:01:27.023072 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.6.47.129:40291)
0810 23:01:27.018640 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.018645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.018826 (+ 181us) server_negotiation.cc:407] Connection header received
0810 23:01:27.018869 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.018871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.018884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.018946 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:27.019193 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020278 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.021334 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.022049 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.022122 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.022182 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.022936 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.022987 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260810 23:01:27.023027 2283 tablet_service.cc:1467] Processing CreateTablet for tablet d66e5137b53048a7b5b369d763aac292 (DEFAULT_TABLE table=test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:27.023286 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36083 (local address 127.0.0.1:35852)
0810 23:01:27.019224 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.019234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:27.019278 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:27.019491 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:27.019492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:27.019495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:27.019531 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.019532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020724 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.020725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.022094 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.022098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.023039 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.023040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.023103 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.023108 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.023168 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.023210 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":67,"threads_started":1}
I20260810 23:01:27.023453 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d66e5137b53048a7b5b369d763aac292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:27.023552 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.018719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.6.47.130:36083)
0810 23:01:27.018927 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.018931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.019333 (+ 402us) server_negotiation.cc:407] Connection header received
0810 23:01:27.019379 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.019382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.019398 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.019473 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:27.019601 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.020616 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.022194 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.022966 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.023035 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.023089 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.023430 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.023477 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260810 23:01:27.023761 1747 tablet_service.cc:1467] Processing CreateTablet for tablet d66e5137b53048a7b5b369d763aac292 (DEFAULT_TABLE table=test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:27.024085 1747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d66e5137b53048a7b5b369d763aac292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:27.024966 2030 tablet_service.cc:1467] Processing CreateTablet for tablet d66e5137b53048a7b5b369d763aac292 (DEFAULT_TABLE table=test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:27.025317 2030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d66e5137b53048a7b5b369d763aac292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:27.026993 2443 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap starting.
I20260810 23:01:27.027735 2443 tablet_bootstrap.cc:654] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:27.028276 2443 log.cc:824] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:27.028688 2445 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Bootstrap starting.
I20260810 23:01:27.028842 2452 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Bootstrap starting.
I20260810 23:01:27.029202 2443 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: No bootstrap required, opened a new log
I20260810 23:01:27.029305 2443 ts_tablet_manager.cc:1397] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:01:27.029577 2452 tablet_bootstrap.cc:654] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:27.030270 2452 log.cc:824] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:27.030304 2445 tablet_bootstrap.cc:654] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:27.030903 2445 log.cc:824] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:27.031178 2452 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: No bootstrap required, opened a new log
I20260810 23:01:27.031261 2452 ts_tablet_manager.cc:1397] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:01:27.031469 2443 raft_consensus.cc:348] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.031648 2443 raft_consensus.cc:374] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:27.031692 2443 raft_consensus.cc:729] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6407f8fc83b4f10a8cbac7c1410d465, State: Initialized, Role: FOLLOWER
I20260810 23:01:27.031728 2445 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: No bootstrap required, opened a new log
I20260810 23:01:27.031803 2445 ts_tablet_manager.cc:1397] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:01:27.031826 2443 consensus_queue.cc:260] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.032126 1879 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:27.032117 2443 ts_tablet_manager.cc:1428] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:01:27.033489 2452 raft_consensus.cc:348] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.033700 2452 raft_consensus.cc:374] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:27.033751 2452 raft_consensus.cc:729] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c9fcf9046a4389b9cb2db7af69a476, State: Initialized, Role: FOLLOWER
I20260810 23:01:27.033900 2452 consensus_queue.cc:260] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.034209 2452 ts_tablet_manager.cc:1428] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:01:27.034096 2445 raft_consensus.cc:348] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.034246 2150 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:27.034274 2445 raft_consensus.cc:374] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:27.034319 2445 raft_consensus.cc:729] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bef7fa8d2f143f8b580a4d41bd4c89a, State: Initialized, Role: FOLLOWER
I20260810 23:01:27.034464 2445 consensus_queue.cc:260] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.034785 2445 ts_tablet_manager.cc:1428] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:01:27.034873 2414 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:27.046581 2459 raft_consensus.cc:482] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:27.046706 2459 raft_consensus.cc:504] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.047117 2459 leader_election.cc:288] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
I20260810 23:01:27.051033 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.047516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:53359 (local address 127.6.47.130:36083)
0810 23:01:27.047578 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.047580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.047594 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:01:27.047637 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.047637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.047645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.047691 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:27.047841 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.048594 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050096 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.050670 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050743 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.050752 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.050927 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.050965 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:01:27.051033 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.047401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:34297 (local address 127.6.47.131:43877)
0810 23:01:27.047470 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.047472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.047483 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:01:27.047512 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.047513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.047526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.047592 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:27.047826 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.048601 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050096 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.050664 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050743 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.050752 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.050928 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.050965 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 23:01:27.051038 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.047283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:43877 (local address 127.6.47.129:34297)
0810 23:01:27.047368 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.047380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:27.047422 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:27.047643 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:27.047644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:27.047646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:27.047721 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.047723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.048691 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.048692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.050001 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.050004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.050755 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.050843 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.050845 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.050904 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.050945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":39}
I20260810 23:01:27.051046 2463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.047300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36083 (local address 127.6.47.129:53359)
0810 23:01:27.047529 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.047540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:27.047586 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:27.047713 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:27.047714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:27.047715 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:27.047772 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.047774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.048691 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.048692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.050001 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:27.050004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.050755 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.050756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:27.050839 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.050844 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:01:27.050904 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:01:27.050945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":89,"thread_start_us":130,"threads_started":1}
I20260810 23:01:27.051390 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d66e5137b53048a7b5b369d763aac292" candidate_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" is_pre_election: true
I20260810 23:01:27.051549 2325 raft_consensus.cc:2454] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6407f8fc83b4f10a8cbac7c1410d465 in term 0.
I20260810 23:01:27.051573 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d66e5137b53048a7b5b369d763aac292" candidate_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" is_pre_election: true
I20260810 23:01:27.051733 2058 raft_consensus.cc:2454] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6407f8fc83b4f10a8cbac7c1410d465 in term 0.
I20260810 23:01:27.051848 1655 leader_election.cc:302] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, f6407f8fc83b4f10a8cbac7c1410d465; no voters:
I20260810 23:01:27.051983 2459 raft_consensus.cc:2781] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:27.052042 2459 raft_consensus.cc:482] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:27.052066 2459 raft_consensus.cc:3037] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:27.052856 2459 raft_consensus.cc:504] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.053001 2459 leader_election.cc:288] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 1 election: Requested vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
I20260810 23:01:27.053160 2325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d66e5137b53048a7b5b369d763aac292" candidate_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
I20260810 23:01:27.053148 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d66e5137b53048a7b5b369d763aac292" candidate_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
I20260810 23:01:27.053249 2325 raft_consensus.cc:3037] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:27.053246 2058 raft_consensus.cc:3037] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:27.054078 2325 raft_consensus.cc:2454] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6407f8fc83b4f10a8cbac7c1410d465 in term 1.
I20260810 23:01:27.054109 2058 raft_consensus.cc:2454] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6407f8fc83b4f10a8cbac7c1410d465 in term 1.
I20260810 23:01:27.054301 1656 leader_election.cc:302] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2c9fcf9046a4389b9cb2db7af69a476, f6407f8fc83b4f10a8cbac7c1410d465; no voters:
I20260810 23:01:27.054415 2459 raft_consensus.cc:2781] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:27.054569 2459 raft_consensus.cc:686] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 LEADER]: Becoming Leader. State: Replica: f6407f8fc83b4f10a8cbac7c1410d465, State: Running, Role: LEADER
I20260810 23:01:27.054669 2459 consensus_queue.cc:237] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:27.055480 1495 catalog_manager.cc:5495] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129). New cstate: current_term: 1 leader_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } health_report { overall_health: UNKNOWN } } }
I20260810 23:01:27.067589 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.065161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.6.47.129:40291)
0810 23:01:27.065228 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:27.065230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:27.065239 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:01:27.065296 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:27.065297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:27.065318 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:27.065399 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:27.065552 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.066029 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.066809 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:27.067007 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:27.067085 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:27.067261 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:27.067362 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:27.067458 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:27.067477 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 23:01:27.067497 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:01:27.069624 2325 raft_consensus.cc:1261] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Refusing update from remote peer f6407f8fc83b4f10a8cbac7c1410d465: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:27.069624 2058 raft_consensus.cc:1261] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Refusing update from remote peer f6407f8fc83b4f10a8cbac7c1410d465: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:27.070022 2466 consensus_queue.cc:1035] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:27.070144 2459 consensus_queue.cc:1035] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:27.072566 2469 mvcc.cc:205] Tried to move back new op lower bound from 7317106225434062848 to 7317106225376534528. Current Snapshot: MvccSnapshot[applied={T|T < 7317106225434062848}]
I20260810 23:01:27.073083 2471 mvcc.cc:205] Tried to move back new op lower bound from 7317106225434062848 to 7317106225376534528. Current Snapshot: MvccSnapshot[applied={T|T < 7317106225434062848}]
I20260810 23:01:27.073390 2473 mvcc.cc:205] Tried to move back new op lower bound from 7317106225434062848 to 7317106225376534528. Current Snapshot: MvccSnapshot[applied={T|T < 7317106225434062848}]
I20260810 23:01:27.075264 1496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:27.075361 1496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:27.076140 1496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Sending DeleteTablet for 3 replicas of tablet d66e5137b53048a7b5b369d763aac292
I20260810 23:01:27.076484 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet d66e5137b53048a7b5b369d763aac292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:27 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:36176
I20260810 23:01:27.076529 2030 tablet_service.cc:1514] Processing DeleteTablet for tablet d66e5137b53048a7b5b369d763aac292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:27 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:35852
I20260810 23:01:27.076545 1747 tablet_service.cc:1514] Processing DeleteTablet for tablet d66e5137b53048a7b5b369d763aac292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:27 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:44114
I20260810 23:01:27.076777 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 1626
I20260810 23:01:27.076871 2489 tablet_replica.cc:330] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: stopping tablet replica
I20260810 23:01:27.076957 2489 raft_consensus.cc:2229] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:27.077040 2489 raft_consensus.cc:2258] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:27.083070 2489 ts_tablet_manager.cc:1905] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:27.088447 2489 ts_tablet_manager.cc:1918] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:27.088575 2489 log.cc:1196] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/d66e5137b53048a7b5b369d763aac292
I20260810 23:01:27.088881 2489 ts_tablet_manager.cc:1939] T d66e5137b53048a7b5b369d763aac292 P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting consensus metadata
I20260810 23:01:27.089293 1469 catalog_manager.cc:4841] TS 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): tablet d66e5137b53048a7b5b369d763aac292 (table test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]) successfully deleted
I20260810 23:01:27.090716 2488 tablet_replica.cc:330] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: stopping tablet replica
I20260810 23:01:27.090842 2488 raft_consensus.cc:2229] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:27.090956 2488 raft_consensus.cc:2258] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:27.091413 2488 ts_tablet_manager.cc:1905] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:27.092823 2488 ts_tablet_manager.cc:1918] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:27.092933 2488 log.cc:1196] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/d66e5137b53048a7b5b369d763aac292
I20260810 23:01:27.093226 2488 ts_tablet_manager.cc:1939] T d66e5137b53048a7b5b369d763aac292 P b2c9fcf9046a4389b9cb2db7af69a476: Deleting consensus metadata
I20260810 23:01:27.093585 1474 catalog_manager.cc:4841] TS b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): tablet d66e5137b53048a7b5b369d763aac292 (table test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]) successfully deleted
W20260810 23:01:27.095867 1474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:40291 (error 108)
I20260810 23:01:27.096210 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 1886
I20260810 23:01:27.099169 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:27.096363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:40291 (local address 127.0.0.1:44130)
0810 23:01:27.099080 (+ 2717us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:27.099128 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2693}
W20260810 23:01:27.101035 1474 catalog_manager.cc:4584] TS f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d66e5137b53048a7b5b369d763aac292: Network error: Client connection negotiation failed: client connection to 127.6.47.129:40291: connect: Connection refused (error 111)
I20260810 23:01:27.105098 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 2158
I20260810 23:01:27.119769 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 1438
I20260810 23:01:47.129938 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--webserver_interface=127.6.47.190
--webserver_port=37867
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:01:47.214582 2494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:47.214784 2494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:47.214807 2494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:47.215914 2494 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:01:47.215951 2494 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:01:47.215965 2494 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:01:47.215977 2494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:01:47.215989 2494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:47.216002 2494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:47.216015 2494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:01:47.216029 2494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:01:47.217585 2494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=37867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:47.217813 2494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:47.217975 2494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:47.227279 2510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.227300 2508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.227291 2514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:47.227710 2494 server_base.cc:1034] running on GCE node
I20260810 23:01:47.228035 2494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:47.228351 2494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:47.229511 2494 hybrid_clock.cc:648] HybridClock initialized: now 1786402907229499 us; error 27 us; skew 500 ppm
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:47.240769 2494 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:01:47.241269 2494 webserver.cc:460] Webserver started at http://127.6.47.190:37867/ using document root <none> and password file <none>
I20260810 23:01:47.241529 2494 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:47.241624 2494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:47.243063 2494 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:01:47.243780 2523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.243961 2494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:01:47.244091 2494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "d35d8be73ac84a99b49e4d091c8a6baa"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "f7cda2f0bdb31633955204d3388fac42"
server_key_iv: "09d322a312636eb99a0fbdf9d38aa737"
server_key_version: "encryptionkey@0"
I20260810 23:01:47.244560 2494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:47.254743 2494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:47.256644 2494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:47.256805 2494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:47.261051 2494 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:38981
I20260810 23:01:47.261087 2615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:38981 every 8 connection(s)
I20260810 23:01:47.261513 2494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:01:47.262974 2616 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:01:47.263607 2616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap starting.
I20260810 23:01:47.266718 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 2494
I20260810 23:01:47.267047 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:40291
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=33175
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:47.273532 2616 log.cc:824] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:47.275421 2616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:47.275743 2616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap complete.
I20260810 23:01:47.278167 2616 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:47.278380 2616 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Initialized, Role: FOLLOWER
I20260810 23:01:47.278551 2616 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:47.278611 2616 raft_consensus.cc:388] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:01:47.278674 2616 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:01:47.278719 2616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 1 FOLLOWER]: Advancing to term 2
I20260810 23:01:47.279753 2616 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:47.279842 2616 leader_election.cc:302] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d35d8be73ac84a99b49e4d091c8a6baa; no voters:
I20260810 23:01:47.280048 2616 leader_election.cc:288] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 23:01:47.280128 2621 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Leader election won for term 2
I20260810 23:01:47.280287 2621 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 LEADER]: Becoming Leader. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Running, Role: LEADER
I20260810 23:01:47.280334 2616 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:01:47.280452 2621 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:01:47.280822 2622 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:01:47.280892 2622 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:47.281148 2623 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35d8be73ac84a99b49e4d091c8a6baa. Latest consensus state: current_term: 2 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:01:47.281212 2623 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:01:47.281424 2628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:01:47.282382 2628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:01:47.282552 2628 catalog_manager.cc:1259] Loaded cluster ID: 6fac4ba7eacd496c93bd8ec9bfc7ec76
I20260810 23:01:47.282753 2628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:01:47.283509 2628 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:01:47.283638 2628 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Loaded TSK: 0
I20260810 23:01:47.295549 2628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Generated new TSK 1
I20260810 23:01:47.295670 2628 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:01:47.366740 2618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:47.366971 2618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:47.367010 2618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:47.368103 2618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:47.368166 2618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:47.368209 2618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:01:47.369752 2618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:40291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=33175
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:47.370018 2618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:47.370254 2618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:47.375388 2668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:47.375622 2618 server_base.cc:1034] running on GCE node
W20260810 23:01:47.375396 2675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.375402 2667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:47.375948 2618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:47.376199 2618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:47.377364 2618 hybrid_clock.cc:648] HybridClock initialized: now 1786402907377346 us; error 35 us; skew 500 ppm
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:47.379834 2618 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:01:47.380302 2618 webserver.cc:460] Webserver started at http://127.6.47.129:33175/ using document root <none> and password file <none>
I20260810 23:01:47.380628 2618 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:47.380724 2618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:47.382226 2618 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:47.382954 2684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.383144 2618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:01:47.383236 2618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "838b82c51197e15772a47f4121c3dfc1"
server_key_iv: "3f5a399436dfc0496b3f02baacfb4057"
server_key_version: "encryptionkey@0"
I20260810 23:01:47.383569 2618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:47.397825 2618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:47.399876 2618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:47.400056 2618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:47.400295 2618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:47.400836 2700 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:01:47.402796 2618 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:01:47.402858 2618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:01:47.402904 2618 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:01:47.403597 2618 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:01:47.403654 2618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:01:47.403748 2700 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap starting.
I20260810 23:01:47.411588 2618 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40291
I20260810 23:01:47.411978 2618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:01:47.412659 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40291 every 8 connection(s)
I20260810 23:01:47.413362 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 2618
I20260810 23:01:47.413640 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:36083
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=34645
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:47.415545 2700 log.cc:824] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:47.418766 2700 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:01:47.419132 2700 tablet_bootstrap.cc:492] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap complete.
I20260810 23:01:47.420406 2700 ts_tablet_manager.cc:1397] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260810 23:01:47.421988 2700 raft_consensus.cc:348] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:47.422345 2700 raft_consensus.cc:729] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6407f8fc83b4f10a8cbac7c1410d465, State: Initialized, Role: FOLLOWER
I20260810 23:01:47.422524 2700 consensus_queue.cc:260] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } }
I20260810 23:01:47.422847 2700 ts_tablet_manager.cc:1428] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:47.428067 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.412689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:53693)
0810 23:01:47.415869 (+ 3180us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.415916 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.416695 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.417312 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.417317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.417673 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:47.417921 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.417927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.419588 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.419589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.420883 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.420885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.421785 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.421786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.421913 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.422522 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:47.422531 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:47.424143 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:47.426291 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.426299 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.426301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.426563 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.426921 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.426922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.426923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.427039 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.427428 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:47.427436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:47.427578 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.427684 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.427869 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3136,"thread_start_us":128,"threads_started":1}
I20260810 23:01:47.428079 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.413076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:53693 (local address 127.6.47.190:38981)
0810 23:01:47.415537 (+ 2461us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.415547 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.416052 (+ 505us) server_negotiation.cc:407] Connection header received
0810 23:01:47.416938 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.416946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.417004 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.417158 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:47.418071 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.419429 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.420979 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.421575 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.421667 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.424393 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:47.424398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:47.424398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:47.424411 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:47.426200 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.426727 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.426730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.426730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.426857 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.427124 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.427125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.427125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.427350 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:47.427448 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.427755 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.427877 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2387,"thread_start_us":87,"threads_started":1}
I20260810 23:01:47.429214 2909 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:47.429327 2909 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:47.429504 2909 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:47.430346 2556 ts_manager.cc:194] Registered new tserver with Master: f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291)
I20260810 23:01:47.431476 2556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:53693
I20260810 23:01:47.432292 2909 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:47.439533 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.431195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.6.47.129:40291)
0810 23:01:47.431361 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.431366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.436732 (+ 5366us) server_negotiation.cc:407] Connection header received
0810 23:01:47.436826 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.436828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.436841 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.436914 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:47.437075 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.437704 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.438666 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.439115 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.439214 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.439270 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.439432 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.439467 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260810 23:01:47.439533 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.430937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:40291 (local address 127.0.0.1:52396)
0810 23:01:47.436623 (+ 5686us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.436654 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.436749 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.436925 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.436928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.436929 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:47.436986 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.436988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.437789 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.437791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.438544 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.438547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.439219 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.439221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.439302 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.439315 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.439412 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.439466 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5659,"thread_start_us":85,"threads_started":1}
I20260810 23:01:47.440239 2778 tablet_service.cc:1514] Processing DeleteTablet for tablet d66e5137b53048a7b5b369d763aac292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:27 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:52396
I20260810 23:01:47.440977 2933 tablet_replica.cc:330] stopping tablet replica
I20260810 23:01:47.441131 2933 raft_consensus.cc:2229] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:47.441205 2933 raft_consensus.cc:2258] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:47.441535 2933 ts_tablet_manager.cc:1905] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:47.443691 2933 ts_tablet_manager.cc:1918] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:47.443781 2933 log.cc:1196] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal/wals/d66e5137b53048a7b5b369d763aac292
I20260810 23:01:47.444029 2933 ts_tablet_manager.cc:1939] T d66e5137b53048a7b5b369d763aac292 P f6407f8fc83b4f10a8cbac7c1410d465: Deleting consensus metadata
I20260810 23:01:47.444573 2526 catalog_manager.cc:4841] TS f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291): tablet d66e5137b53048a7b5b369d763aac292 (table test-table [id=4f1cf7429d8542f9bbe106bcbaf2adf4]) successfully deleted
W20260810 23:01:47.506772 2926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:47.506973 2926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:47.506999 2926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:47.508059 2926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:47.508091 2926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:47.508117 2926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:01:47.509610 2926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:36083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=34645
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:47.509816 2926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:47.510057 2926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:47.515142 2946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.515187 2944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.515143 2943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:47.515460 2926 server_base.cc:1034] running on GCE node
I20260810 23:01:47.515671 2926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:47.515914 2926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:47.517078 2926 hybrid_clock.cc:648] HybridClock initialized: now 1786402907517064 us; error 32 us; skew 500 ppm
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:47.520359 2926 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:01:47.521031 2926 webserver.cc:460] Webserver started at http://127.6.47.130:34645/ using document root <none> and password file <none>
I20260810 23:01:47.521315 2926 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:47.521437 2926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:47.522783 2926 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:01:47.523373 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.523577 2926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:01:47.523681 2926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "26b275414d5c580003a067dc7c3d2ed9"
server_key_iv: "954479d94d0705f734875b91596dcbfc"
server_key_version: "encryptionkey@0"
I20260810 23:01:47.524034 2926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:47.546041 2926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:47.548236 2926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:47.548431 2926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:47.548699 2926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:47.549011 2926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:47.549065 2926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.549104 2926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:47.549130 2926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.555840 2926 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:36083
I20260810 23:01:47.555891 3171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:36083 every 8 connection(s)
I20260810 23:01:47.556218 2926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:01:47.562126 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 2926
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:47.562460 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:43877
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=46073
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:01:47.566504 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.557042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:41403 (local address 127.6.47.190:38981)
0810 23:01:47.557097 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.557100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.557380 (+ 280us) server_negotiation.cc:407] Connection header received
0810 23:01:47.557831 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.557832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.557847 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.557917 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:47.558803 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.559361 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.560286 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.560547 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.560635 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.562986 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:47.562988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:47.562988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:47.562996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:47.564722 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.565296 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.565297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.565297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.565347 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.565737 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.565738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.565738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.565832 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:47.565881 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.566278 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.566370 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:01:47.566608 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.557053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:41403)
0810 23:01:47.557251 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.557288 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.557742 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.557968 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.557971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.558530 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:47.558705 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.558710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.559453 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.559454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.560203 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.560205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.560645 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.560645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.560773 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.561231 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:47.561239 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:47.562875 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:47.564819 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.564828 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.564831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.565204 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.565422 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.565423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.565424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.565659 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.565906 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:47.565924 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:47.566099 (+ 175us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.566234 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.566405 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260810 23:01:47.567224 3172 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:47.567356 3172 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:47.567554 3172 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:47.568035 2556 ts_manager.cc:194] Registered new tserver with Master: b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
I20260810 23:01:47.568454 2556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:41403
W20260810 23:01:47.648455 3178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:01:47.648720 3178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:01:47.648759 3178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:01:47.650027 3178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:01:47.650097 3178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:01:47.650142 3178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:01:47.651598 3178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:43877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=46073
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:01:47.651889 3178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:01:47.652124 3178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:01:47.657286 3190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.657270 3198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:01:47.657315 3191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:01:47.657450 3178 server_base.cc:1034] running on GCE node
I20260810 23:01:47.657662 3178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:01:47.657889 3178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:01:47.659025 3178 hybrid_clock.cc:648] HybridClock initialized: now 1786402907658979 us; error 62 us; skew 500 ppm
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:01:47.661846 3178 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:01:47.662325 3178 webserver.cc:460] Webserver started at http://127.6.47.131:46073/ using document root <none> and password file <none>
I20260810 23:01:47.662546 3178 fs_manager.cc:359] Metadata directory not provided
I20260810 23:01:47.662623 3178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:01:47.663913 3178 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:47.664561 3209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.664755 3178 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:01:47.664860 3178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "ddbd718681476a1dee7defdb20fc7d17"
server_key_iv: "bc34fe48065a52eac2067634e33c15e9"
server_key_version: "encryptionkey@0"
I20260810 23:01:47.665212 3178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:01:47.682651 3178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:01:47.684583 3178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:01:47.684744 3178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:01:47.684988 3178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:01:47.685302 3178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:01:47.685357 3178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.685402 3178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:01:47.685427 3178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.691790 3178 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:43877
I20260810 23:01:47.691834 3414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:43877 every 8 connection(s)
I20260810 23:01:47.692211 3178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
Aug 10 23:01:47 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402907, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:01:47.699661 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 3178
I20260810 23:01:47.701718 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.692962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:53999 (local address 127.6.47.190:38981)
0810 23:01:47.693034 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.693035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.693270 (+ 235us) server_negotiation.cc:407] Connection header received
0810 23:01:47.693735 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.693735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.693745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.693802 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:47.694727 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.695236 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.696209 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.696399 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.696449 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.698755 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:47.698756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:47.698757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:47.698763 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:47.700165 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.700708 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.700710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.700710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.700771 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.701038 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.701039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.701040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.701203 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:47.701271 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.701536 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.701625 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:01:47.701802 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.692945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:53999)
0810 23:01:47.693134 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.693177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.693651 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.693870 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.693873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.694460 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:01:47.694646 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.694651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.695328 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.695329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.696128 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.696131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.696472 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.696473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.696648 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.697133 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:01:47.697142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:01:47.698491 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:01:47.700246 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.700253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.700256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.700622 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.700847 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:01:47.700848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:01:47.700849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:01:47.700958 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:01:47.701278 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:01:47.701286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:01:47.701406 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.701492 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.701610 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260810 23:01:47.702322 3415 heartbeater.cc:347] Connected to a master server at 127.6.47.190:38981
I20260810 23:01:47.702436 3415 heartbeater.cc:464] Registering TS with master...
I20260810 23:01:47.702615 3415 heartbeater.cc:511] Master 127.6.47.190:38981 requested a full tablet report, sending...
I20260810 23:01:47.703086 2556 ts_manager.cc:194] Registered new tserver with Master: 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877)
I20260810 23:01:47.703577 2556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:53999
I20260810 23:01:47.707942 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.700140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.6.47.190:38981)
0810 23:01:47.700310 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.700311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.700379 (+ 68us) server_negotiation.cc:407] Connection header received
0810 23:01:47.700420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.700421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.700433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.700525 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:01:47.701114 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.701803 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.702919 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.703236 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.703299 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.704599 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:01:47.704601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:01:47.704601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:01:47.704607 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:01:47.706569 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.707018 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.707018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.707019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.707094 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:01:47.707336 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:01:47.707337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:01:47.707337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:01:47.707476 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:01:47.707704 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.707752 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.707842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260810 23:01:47.709019 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:01:47.714874 3421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.710064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.6.47.190:38981)
0810 23:01:47.710119 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.710121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.710152 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:01:47.710190 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.710191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.710230 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.710332 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:47.710556 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.711224 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.712400 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.712729 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.712812 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.713730 (+ 918us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:47.713861 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:47.713950 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.714775 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.714798 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:01:47.715258 2556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:01:47.716966 2556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:01:47.719396 2778 tablet_service.cc:1467] Processing CreateTablet for tablet 760922349aab4eb0bc62c7ed705854ab (DEFAULT_TABLE table=test-table [id=c4d6acb40de14bc5b8e656b68c22a20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:47.719610 2778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760922349aab4eb0bc62c7ed705854ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:47.721172 2700 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap starting.
I20260810 23:01:47.721511 2700 tablet_bootstrap.cc:654] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:47.722157 2700 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: No bootstrap required, opened a new log
I20260810 23:01:47.722210 2700 ts_tablet_manager.cc:1397] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260810 23:01:47.722337 2700 raft_consensus.cc:348] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.722419 2700 raft_consensus.cc:374] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:47.722471 2700 raft_consensus.cc:729] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6407f8fc83b4f10a8cbac7c1410d465, State: Initialized, Role: FOLLOWER
I20260810 23:01:47.722519 2700 consensus_queue.cc:260] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.722613 2700 ts_tablet_manager.cc:1428] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260810 23:01:47.723199 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.718861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:43877 (local address 127.0.0.1:38042)
0810 23:01:47.718911 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.718923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.718964 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.719471 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.719472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.719474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:47.719513 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.719515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.720761 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.720762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.721774 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.721778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.722954 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.722955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.723011 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.723015 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.723047 (+ 32us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.723100 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 23:01:47.723423 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.718922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36083 (local address 127.0.0.1:60148)
0810 23:01:47.719075 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.719101 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.719154 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.719733 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.719734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.719736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:47.719858 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.719860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.721072 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.721073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.722212 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.722215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.723176 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.723177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.723237 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.723241 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.723315 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.723353 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260810 23:01:47.723439 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.719279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.6.47.130:36083)
0810 23:01:47.719476 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.719481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.719492 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:01:47.719532 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.719534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.719550 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.719625 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:47.719936 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.720985 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.722315 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.723040 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.723178 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.723240 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.723329 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.723364 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260810 23:01:47.723654 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.718966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38042 (local address 127.6.47.131:43877)
0810 23:01:47.719304 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.719308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.719317 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:01:47.719350 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.719351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.719363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.719470 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:47.719574 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.720674 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.722025 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.722793 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.723381 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.723447 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.723495 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.723543 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":157,"threads_started":1}
I20260810 23:01:47.724131 3040 tablet_service.cc:1467] Processing CreateTablet for tablet 760922349aab4eb0bc62c7ed705854ab (DEFAULT_TABLE table=test-table [id=c4d6acb40de14bc5b8e656b68c22a20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:47.724257 3296 tablet_service.cc:1467] Processing CreateTablet for tablet 760922349aab4eb0bc62c7ed705854ab (DEFAULT_TABLE table=test-table [id=c4d6acb40de14bc5b8e656b68c22a20d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:01:47.724541 3040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760922349aab4eb0bc62c7ed705854ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:47.724646 3296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 760922349aab4eb0bc62c7ed705854ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:01:47.727463 3431 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Bootstrap starting.
I20260810 23:01:47.727617 3433 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Bootstrap starting.
I20260810 23:01:47.728225 3431 tablet_bootstrap.cc:654] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:47.728225 3433 tablet_bootstrap.cc:654] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Neither blocks nor log segments found. Creating new log.
I20260810 23:01:47.729001 3431 log.cc:824] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:47.729001 3433 log.cc:824] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Log is configured to *not* fsync() on all Append() calls
I20260810 23:01:47.729750 3433 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: No bootstrap required, opened a new log
I20260810 23:01:47.729758 3431 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: No bootstrap required, opened a new log
I20260810 23:01:47.729807 3433 ts_tablet_manager.cc:1397] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:01:47.729838 3431 ts_tablet_manager.cc:1397] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:01:47.731158 3433 raft_consensus.cc:348] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.731340 3433 raft_consensus.cc:374] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:47.731397 3433 raft_consensus.cc:729] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c9fcf9046a4389b9cb2db7af69a476, State: Initialized, Role: FOLLOWER
I20260810 23:01:47.731220 3431 raft_consensus.cc:348] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.731535 3431 raft_consensus.cc:374] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:01:47.731545 3433 consensus_queue.cc:260] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.731609 3431 raft_consensus.cc:729] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bef7fa8d2f143f8b580a4d41bd4c89a, State: Initialized, Role: FOLLOWER
I20260810 23:01:47.731761 3431 consensus_queue.cc:260] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.731863 3433 ts_tablet_manager.cc:1428] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:01:47.731886 3172 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:47.732067 3431 ts_tablet_manager.cc:1428] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:01:47.732116 3415 heartbeater.cc:503] Master 127.6.47.190:38981 was elected leader, sending a full tablet report...
I20260810 23:01:47.764351 3442 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:01:47.764479 3442 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.764922 3442 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083), f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291)
I20260810 23:01:47.767722 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.765129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36083 (local address 127.6.47.131:38865)
0810 23:01:47.765190 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.765204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.765253 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.765459 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.765460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.765461 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:47.765538 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.765539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.766168 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.766169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.766887 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.766889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.767452 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.767454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.767544 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.767548 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.767610 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.767650 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 23:01:47.767733 3427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.765200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:38865 (local address 127.6.47.130:36083)
0810 23:01:47.765248 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.765250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.765260 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:01:47.765351 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.765352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.765363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.765416 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:47.765618 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.766080 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.766979 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.767367 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.767468 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.767476 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.767636 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.767674 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:01:47.768083 3073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760922349aab4eb0bc62c7ed705854ab" candidate_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" is_pre_election: true
I20260810 23:01:47.768157 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.765253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:40291 (local address 127.6.47.131:55635)
0810 23:01:47.765449 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.765465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 23:01:47.765547 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:01:47.765721 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:01:47.765722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:01:47.765723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:01:47.765801 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.765802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.766410 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.766411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.767136 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:01:47.767137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.767896 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.767898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:01:47.767974 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.767979 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:01:47.768041 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 23:01:47.768078 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":123,"threads_started":1}
I20260810 23:01:47.768162 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.765363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:55635 (local address 127.6.47.129:40291)
0810 23:01:47.765425 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.765427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.765533 (+ 106us) server_negotiation.cc:407] Connection header received
0810 23:01:47.765605 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.765607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.765618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.765710 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:01:47.765871 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.766319 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.767247 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.767740 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.767830 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.767841 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.768068 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.768116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:01:47.768263 3073 raft_consensus.cc:2454] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bef7fa8d2f143f8b580a4d41bd4c89a in term 0.
I20260810 23:01:47.768587 3212 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476; no voters:
I20260810 23:01:47.768648 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760922349aab4eb0bc62c7ed705854ab" candidate_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" is_pre_election: true
I20260810 23:01:47.768757 3442 raft_consensus.cc:2781] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:01:47.768816 3442 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:01:47.768816 2823 raft_consensus.cc:2454] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bef7fa8d2f143f8b580a4d41bd4c89a in term 0.
I20260810 23:01:47.768851 3442 raft_consensus.cc:3037] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:47.769578 3442 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.769731 3442 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [CANDIDATE]: Term 1 election: Requested vote from peers b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083), f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291)
I20260810 23:01:47.769881 3073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760922349aab4eb0bc62c7ed705854ab" candidate_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
I20260810 23:01:47.769906 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "760922349aab4eb0bc62c7ed705854ab" candidate_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
I20260810 23:01:47.769976 3073 raft_consensus.cc:3037] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:47.769982 2823 raft_consensus.cc:3037] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:01:47.770677 3073 raft_consensus.cc:2454] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bef7fa8d2f143f8b580a4d41bd4c89a in term 1.
I20260810 23:01:47.770908 3212 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476; no voters:
I20260810 23:01:47.771006 3442 raft_consensus.cc:2781] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:01:47.771152 3442 raft_consensus.cc:686] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 LEADER]: Becoming Leader. State: Replica: 0bef7fa8d2f143f8b580a4d41bd4c89a, State: Running, Role: LEADER
I20260810 23:01:47.771298 2823 raft_consensus.cc:2454] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bef7fa8d2f143f8b580a4d41bd4c89a in term 1.
I20260810 23:01:47.771265 3442 consensus_queue.cc:237] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:01:47.772063 2551 catalog_manager.cc:5495] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131). New cstate: current_term: 1 leader_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } health_report { overall_health: UNKNOWN } } }
I20260810 23:01:47.782361 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.779676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.6.47.131:43877)
0810 23:01:47.779719 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:01:47.779720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:01:47.779727 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:01:47.779816 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:01:47.779816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:01:47.779832 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:01:47.779884 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:01:47.779970 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.780583 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.781393 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:01:47.781758 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:01:47.781820 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:01:47.781984 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:01:47.782105 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:01:47.782191 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 23:01:47.782234 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 23:01:47.782256 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260810 23:01:47.784587 2823 raft_consensus.cc:1261] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Refusing update from remote peer 0bef7fa8d2f143f8b580a4d41bd4c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:47.784617 3073 raft_consensus.cc:1261] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Refusing update from remote peer 0bef7fa8d2f143f8b580a4d41bd4c89a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:01:47.785015 3442 consensus_queue.cc:1035] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:47.785174 3446 consensus_queue.cc:1035] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:01:47.787636 3450 mvcc.cc:205] Tried to move back new op lower bound from 7317106310282493952 to 7317106310231732224. Current Snapshot: MvccSnapshot[applied={T|T < 7317106310282493952}]
I20260810 23:01:47.789757 2556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:47.789867 2556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:01:47.790584 2556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Sending DeleteTablet for 3 replicas of tablet 760922349aab4eb0bc62c7ed705854ab
I20260810 23:01:47.790887 2778 tablet_service.cc:1514] Processing DeleteTablet for tablet 760922349aab4eb0bc62c7ed705854ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:47 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:52396
I20260810 23:01:47.790901 3296 tablet_service.cc:1514] Processing DeleteTablet for tablet 760922349aab4eb0bc62c7ed705854ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:47 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:38042
I20260810 23:01:47.790897 3040 tablet_service.cc:1514] Processing DeleteTablet for tablet 760922349aab4eb0bc62c7ed705854ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:01:47 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:60148
I20260810 23:01:47.790977 2933 tablet_replica.cc:330] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: stopping tablet replica
I20260810 23:01:47.791044 2933 raft_consensus.cc:2229] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:47.791096 2933 raft_consensus.cc:2258] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:47.791345 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 2618
I20260810 23:01:47.791550 2933 ts_tablet_manager.cc:1905] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:47.802868 3468 tablet_replica.cc:330] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: stopping tablet replica
I20260810 23:01:47.803090 3471 tablet_replica.cc:330] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: stopping tablet replica
I20260810 23:01:47.803278 3471 raft_consensus.cc:2229] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:01:47.803421 3468 raft_consensus.cc:2229] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:01:47.803586 3471 raft_consensus.cc:2258] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:47.803740 3468 raft_consensus.cc:2258] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:01:47.804590 3468 ts_tablet_manager.cc:1905] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:47.805859 3471 ts_tablet_manager.cc:1905] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:01:47.806537 3468 ts_tablet_manager.cc:1918] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:47.806669 3468 log.cc:1196] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/760922349aab4eb0bc62c7ed705854ab
I20260810 23:01:47.807052 3468 ts_tablet_manager.cc:1939] T 760922349aab4eb0bc62c7ed705854ab P 0bef7fa8d2f143f8b580a4d41bd4c89a: Deleting consensus metadata
I20260810 23:01:47.807624 2525 catalog_manager.cc:4841] TS 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): tablet 760922349aab4eb0bc62c7ed705854ab (table test-table [id=c4d6acb40de14bc5b8e656b68c22a20d]) successfully deleted
I20260810 23:01:47.807832 3471 ts_tablet_manager.cc:1918] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:01:47.807910 3471 log.cc:1196] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/760922349aab4eb0bc62c7ed705854ab
I20260810 23:01:47.808209 3471 ts_tablet_manager.cc:1939] T 760922349aab4eb0bc62c7ed705854ab P b2c9fcf9046a4389b9cb2db7af69a476: Deleting consensus metadata
I20260810 23:01:47.808624 2526 catalog_manager.cc:4841] TS b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): tablet 760922349aab4eb0bc62c7ed705854ab (table test-table [id=c4d6acb40de14bc5b8e656b68c22a20d]) successfully deleted
W20260810 23:01:47.811038 2526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:40291 (error 108)
I20260810 23:01:47.811349 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 2926
I20260810 23:01:47.811573 3423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:01:47.811437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:40291 (local address 127.0.0.1:52402)
0810 23:01:47.811495 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:01:47.811536 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:01:47.836822 2526 catalog_manager.cc:4584] TS f6407f8fc83b4f10a8cbac7c1410d465 (127.6.47.129:40291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 760922349aab4eb0bc62c7ed705854ab: Network error: Client connection negotiation failed: client connection to 127.6.47.129:40291: connect: Connection refused (error 111)
I20260810 23:01:47.843226 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 3178
I20260810 23:01:47.857081 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 2494
I20260810 23:02:07.870818 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--webserver_interface=127.6.47.190
--webserver_port=37867
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:07.955835 3473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:07.956036 3473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:07.956058 3473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:07.957229 3473 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 23:02:07.957270 3473 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 23:02:07.957283 3473 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 23:02:07.957294 3473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:07.957348 3473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:07.957367 3473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:07.957388 3473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:07.957410 3473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:07.958885 3473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=37867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:07.959162 3473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:07.959414 3473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:07.964738 3480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:07.964779 3479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:07.964795 3473 server_base.cc:1034] running on GCE node
W20260810 23:02:07.964749 3484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:07.965256 3473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:07.965595 3473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:07.966758 3473 hybrid_clock.cc:648] HybridClock initialized: now 1786402927966739 us; error 40 us; skew 500 ppm
Aug 10 23:02:07 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402927, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:07.969321 3473 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:07.969858 3473 webserver.cc:460] Webserver started at http://127.6.47.190:37867/ using document root <none> and password file <none>
I20260810 23:02:07.970088 3473 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:07.970155 3473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:07.971504 3473 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:07.972229 3499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:07.972410 3473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:07.972538 3473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "d35d8be73ac84a99b49e4d091c8a6baa"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "f7cda2f0bdb31633955204d3388fac42"
server_key_iv: "09d322a312636eb99a0fbdf9d38aa737"
server_key_version: "encryptionkey@0"
I20260810 23:02:07.972936 3473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:07.985049 3473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:07.986815 3473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:07.986985 3473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:07.991284 3473 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:38981
I20260810 23:02:07.991319 3603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:38981 every 8 connection(s)
I20260810 23:02:07.991721 3473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:07.993270 3604 sys_catalog.cc:263] Verifying existing consensus state
I20260810 23:02:07.993966 3604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap starting.
I20260810 23:02:07.998023 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 3473
I20260810 23:02:07.998314 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:40291
--local_ip_for_outbound_sockets=127.6.47.129
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=33175
--webserver_interface=127.6.47.129
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:02:08.003705 3604 log.cc:824] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:08.006274 3604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:02:08.006615 3604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: Bootstrap complete.
I20260810 23:02:08.009070 3604 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:02:08.009357 3604 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Initialized, Role: FOLLOWER
I20260810 23:02:08.009554 3604 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:02:08.009644 3604 raft_consensus.cc:388] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:08.009703 3604 raft_consensus.cc:482] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:08.009758 3604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 2 FOLLOWER]: Advancing to term 3
I20260810 23:02:08.010774 3604 raft_consensus.cc:504] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:02:08.010900 3604 leader_election.cc:302] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d35d8be73ac84a99b49e4d091c8a6baa; no voters:
I20260810 23:02:08.011108 3604 leader_election.cc:288] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 23:02:08.011178 3611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 3 FOLLOWER]: Leader election won for term 3
I20260810 23:02:08.011332 3611 raft_consensus.cc:686] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [term 3 LEADER]: Becoming Leader. State: Replica: d35d8be73ac84a99b49e4d091c8a6baa, State: Running, Role: LEADER
I20260810 23:02:08.011410 3604 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:08.011439 3611 consensus_queue.cc:237] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } }
I20260810 23:02:08.011736 3611 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d35d8be73ac84a99b49e4d091c8a6baa. Latest consensus state: current_term: 3 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:02:08.011809 3611 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:08.011736 3614 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d35d8be73ac84a99b49e4d091c8a6baa" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38981 } } }
I20260810 23:02:08.011974 3614 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:08.013293 3646 catalog_manager.cc:1259] Loaded cluster ID: 6fac4ba7eacd496c93bd8ec9bfc7ec76
I20260810 23:02:08.013342 3646 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: loading cluster ID for follower catalog manager: success
I20260810 23:02:08.014492 3646 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d35d8be73ac84a99b49e4d091c8a6baa: acquiring CA information for follower catalog manager: success
F20260810 23:02:08.014894 3646 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff98476f9ad google::LogMessage::Fail() at ??:0
@ 0x7ff98477288a google::LogMessage::SendToLog() at ??:0
@ 0x7ff98476f4de google::LogMessage::Flush() at ??:0
@ 0x7ff984770a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff986edc104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff98c6c86b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7ff98c6c8989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7ff98c6c8e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7ff98c6cb8c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7ff986632954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff9874966db start_thread at ??:0
@ 0x7ff98326d61f clone at ??:0
W20260810 23:02:08.099536 3609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:08.099735 3609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:08.099759 3609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:08.100962 3609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:08.101004 3609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:08.101032 3609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:08.102494 3609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:40291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=33175
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:08.102741 3609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:08.102989 3609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:08.108083 3684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:08.108052 3680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:08.108328 3609 server_base.cc:1034] running on GCE node
W20260810 23:02:08.108048 3678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:08.108629 3609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:08.108918 3609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:08.110078 3609 hybrid_clock.cc:648] HybridClock initialized: now 1786402928110063 us; error 32 us; skew 500 ppm
Aug 10 23:02:08 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402928, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:08.112807 3609 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:02:08.113277 3609 webserver.cc:460] Webserver started at http://127.6.47.129:33175/ using document root <none> and password file <none>
I20260810 23:02:08.113520 3609 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:08.113592 3609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:08.115013 3609 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:08.115877 3704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.116062 3609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:08.116168 3609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "f6407f8fc83b4f10a8cbac7c1410d465"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "838b82c51197e15772a47f4121c3dfc1"
server_key_iv: "3f5a399436dfc0496b3f02baacfb4057"
server_key_version: "encryptionkey@0"
I20260810 23:02:08.116542 3609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:08.134918 3609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:08.137390 3609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:08.137562 3609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:08.137811 3609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:08.138290 3713 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 23:02:08.140384 3609 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 23:02:08.140448 3609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260810 23:02:08.140522 3609 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 23:02:08.141180 3609 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 23:02:08.141237 3609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260810 23:02:08.141316 3713 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap starting.
I20260810 23:02:08.143307 3713 log.cc:824] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:08.146757 3713 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 23:02:08.147116 3713 tablet_bootstrap.cc:492] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Bootstrap complete.
I20260810 23:02:08.147306 3713 ts_tablet_manager.cc:1397] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 23:02:08.148697 3609 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:40291
I20260810 23:02:08.149091 3609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:08.149432 3914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:40291 every 8 connection(s)
I20260810 23:02:08.149555 3713 raft_consensus.cc:348] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:08.149914 3713 raft_consensus.cc:729] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6407f8fc83b4f10a8cbac7c1410d465, State: Initialized, Role: FOLLOWER
I20260810 23:02:08.150063 3713 consensus_queue.cc:260] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:08.150334 3713 ts_tablet_manager.cc:1428] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260810 23:02:08.154654 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 3609
I20260810 23:02:08.154973 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:36083
--local_ip_for_outbound_sockets=127.6.47.130
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=34645
--webserver_interface=127.6.47.130
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:08.231106 3919 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:02:08.149883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:44545)
0810 23:02:08.150075 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.150117 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:08.150283 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:08.230849 (+ 80566us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
W20260810 23:02:08.231357 3916 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:38981 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:38981: Client connection negotiation failed: client connection to 127.6.47.190:38981: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 23:02:08.231763 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.231634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:56677)
0810 23:02:08.231700 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.231744 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:02:08.232100 3919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.232015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:46395)
0810 23:02:08.232061 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.232085 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 23:02:08.232236 3916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:02:08.263759 3922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:08.263948 3922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:08.263972 3922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:08.265166 3922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:08.265208 3922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:08.265234 3922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:08.266626 3922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:36083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=34645
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:08.266801 3922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:08.266968 3922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:08.272184 3927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:08.272225 3934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:08.272195 3931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:08.272425 3922 server_base.cc:1034] running on GCE node
I20260810 23:02:08.272660 3922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:08.272894 3922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:08.274058 3922 hybrid_clock.cc:648] HybridClock initialized: now 1786402928274033 us; error 50 us; skew 500 ppm
Aug 10 23:02:08 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402928, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:08.276626 3922 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:02:08.277122 3922 webserver.cc:460] Webserver started at http://127.6.47.130:34645/ using document root <none> and password file <none>
I20260810 23:02:08.277309 3922 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:08.277349 3922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:08.278542 3922 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:08.279291 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.279469 3922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:08.279582 3922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "b2c9fcf9046a4389b9cb2db7af69a476"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "26b275414d5c580003a067dc7c3d2ed9"
server_key_iv: "954479d94d0705f734875b91596dcbfc"
server_key_version: "encryptionkey@0"
I20260810 23:02:08.279897 3922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:08.292642 3922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:08.294399 3922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:08.294505 3922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:08.294694 3922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:08.294956 3922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:08.294987 3922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.295008 3922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:08.295022 3922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.300913 3922 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:36083
I20260810 23:02:08.300985 4149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:36083 every 8 connection(s)
I20260810 23:02:08.301306 3922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:02:08.301988 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 3922
I20260810 23:02:08.302301 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:43877
--local_ip_for_outbound_sockets=127.6.47.131
--tserver_master_addrs=127.6.47.190:38981
--webserver_port=46073
--webserver_interface=127.6.47.131
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:41133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:02:08.302474 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.301939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:35635)
0810 23:02:08.302176 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.302269 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
W20260810 23:02:08.302829 4150 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:38981 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:38981: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
I20260810 23:02:08.303154 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.303051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:47139)
0810 23:02:08.303105 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.303135 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:02:08.303429 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.303364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:49039)
0810 23:02:08.303398 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.303417 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260810 23:02:08.303550 4150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 23:02:08.386457 4156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:08.386663 4156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:08.386686 4156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:08.387795 4156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:08.387836 4156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:08.387887 4156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:08.389340 4156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:41133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:43877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=46073
--tserver_master_addrs=127.6.47.190:38981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:08.389570 4156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:08.389793 4156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:08.394860 4168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:08.394855 4170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:08.394853 4175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:08.395112 4156 server_base.cc:1034] running on GCE node
I20260810 23:02:08.395354 4156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:08.395602 4156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:08.396755 4156 hybrid_clock.cc:648] HybridClock initialized: now 1786402928396747 us; error 34 us; skew 500 ppm
Aug 10 23:02:08 dist-test-slave-rwrg krb5kdc[1408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402928, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:08.399227 4156 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:02:08.399624 4156 webserver.cc:460] Webserver started at http://127.6.47.131:46073/ using document root <none> and password file <none>
I20260810 23:02:08.399936 4156 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:08.400024 4156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:08.401412 4156 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:08.401988 4187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.402158 4156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:08.402245 4156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a"
format_stamp: "Formatted at 2026-08-10 23:01:26 on dist-test-slave-rwrg"
server_key: "ddbd718681476a1dee7defdb20fc7d17"
server_key_iv: "bc34fe48065a52eac2067634e33c15e9"
server_key_version: "encryptionkey@0"
I20260810 23:02:08.402551 4156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:08.424459 4156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:08.426420 4156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:08.426585 4156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:08.426851 4156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:08.427184 4156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:08.427237 4156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.427279 4156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:08.427304 4156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:08.433245 4156 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:43877
I20260810 23:02:08.433305 4370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:43877 every 8 connection(s)
I20260810 23:02:08.433645 4156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:08.434615 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.434244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:44087)
0810 23:02:08.434396 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.434467 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
W20260810 23:02:08.435020 4371 heartbeater.cc:644] Failed to heartbeat to 127.6.47.190:38981 (0 consecutive failures): Network error: Failed to ping master at 127.6.47.190:38981: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
I20260810 23:02:08.435344 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.435234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:55979)
0810 23:02:08.435292 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.435326 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:02:08.435680 4373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:08.435603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:58261)
0810 23:02:08.435643 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:08.435665 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260810 23:02:08.435806 4371 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 23:02:08.438441 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 4156
I20260810 23:02:09.232883 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.232652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:47975)
0810 23:02:09.232783 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:09.232819 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260810 23:02:09.304065 4377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.303855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:34421)
0810 23:02:09.303975 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:09.304011 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260810 23:02:09.436333 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.436152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:53287)
0810 23:02:09.436257 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:09.436285 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260810 23:02:09.522236 4380 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:09.522398 4380 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:09.522765 4380 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
Aug 10 23:02:09 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402928, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.131@KRBTEST.COM
Aug 10 23:02:09 dist-test-slave-rwrg krb5kdc[1408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402928, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.130@KRBTEST.COM
I20260810 23:02:09.533545 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.523121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:50975 (local address 127.6.47.130:36083)
0810 23:02:09.523277 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:09.523286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:09.523311 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:02:09.523749 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:09.523751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:09.523791 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:09.523946 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:09.524796 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.525754 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.526845 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.527228 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.527349 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:09.530097 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:09.530101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:09.530101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:09.530110 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:09.531827 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532258 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:09.532261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:09.532261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:09.532346 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532652 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:09.532653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:09.532653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:09.532877 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:09.532961 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:09.533307 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 23:02:09.533430 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260810 23:02:09.533527 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.522935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:36083 (local address 127.6.47.129:50975)
0810 23:02:09.522994 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:09.523008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:09.523080 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:09.523981 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:09.523983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:09.524380 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:09.524661 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:09.524668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.525855 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.525856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:09.526726 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:09.526729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.527325 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.527326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:09.527442 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:09.528073 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:09.528078 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:09.530001 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:09.531915 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:09.531923 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:09.531925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:09.532167 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:09.532419 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:09.532420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:09.532577 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:09.532955 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:09.532963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:09.533088 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 23:02:09.533179 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 23:02:09.533425 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":61}
I20260810 23:02:09.533557 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.523082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:40473 (local address 127.6.47.131:43877)
0810 23:02:09.523252 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:09.523264 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:09.523302 (+ 38us) server_negotiation.cc:407] Connection header received
0810 23:02:09.523759 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:09.523761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:09.523798 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:09.523947 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:09.525016 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.525893 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.526810 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.527224 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.527336 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:09.529773 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:09.529777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:09.529778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:09.530069 (+ 291us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:09.531966 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532335 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:09.532339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:09.532339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:09.532430 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532663 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:09.532664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:09.532664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:09.532884 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:09.532966 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:09.533303 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 23:02:09.533437 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260810 23:02:09.533653 4381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:09.522953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:43877 (local address 127.6.47.129:40473)
0810 23:02:09.523068 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:09.523078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:09.523138 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:09.523981 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:09.523983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:09.524379 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:09.524884 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:09.524888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.525994 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.525994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:09.526725 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:09.526727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:09.527315 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:09.527316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:09.527442 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:09.528015 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:09.528023 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:09.529594 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:09.532072 (+ 2478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:09.532073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:09.532256 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:09.532532 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:09.532533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:09.532533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:09.532593 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:09.532962 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:09.532963 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:09.533088 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 23:02:09.533179 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 23:02:09.533506 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":131,"thread_start_us":55,"threads_started":1}
W20260810 23:02:09.534999 3706 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
W20260810 23:02:09.535185 3707 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
I20260810 23:02:09.535235 3707 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6407f8fc83b4f10a8cbac7c1410d465; no voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476
I20260810 23:02:09.535334 4380 raft_consensus.cc:2726] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:02:10.233587 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:10.233378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:41689)
0810 23:02:10.233503 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:10.233538 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260810 23:02:10.304736 4386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:10.304483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:42743)
0810 23:02:10.304661 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:10.304698 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260810 23:02:10.436899 4387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:10.436717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:34743)
0810 23:02:10.436831 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:10.436864 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260810 23:02:11.234261 4388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:11.234032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:53369)
0810 23:02:11.234175 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:11.234212 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260810 23:02:11.305418 4393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:11.305183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:44651)
0810 23:02:11.305344 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:11.305380 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260810 23:02:11.360491 4394 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:11.360628 4394 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:11.360848 4394 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
W20260810 23:02:11.361375 3707 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
W20260810 23:02:11.361454 3706 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
I20260810 23:02:11.361545 3706 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6407f8fc83b4f10a8cbac7c1410d465; no voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476
I20260810 23:02:11.361649 4394 raft_consensus.cc:2726] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:02:11.437510 4398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:11.437292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:53937)
0810 23:02:11.437434 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:11.437470 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260810 23:02:12.235288 4400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:12.235022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:49209)
0810 23:02:12.235202 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:12.235239 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260810 23:02:12.306062 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:12.305858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:45627)
0810 23:02:12.305982 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:12.306023 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260810 23:02:12.438174 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:12.437953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:38811)
0810 23:02:12.438089 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:12.438126 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260810 23:02:13.081139 4407 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:13.081261 4407 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:13.081465 4407 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
W20260810 23:02:13.081851 3706 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
W20260810 23:02:13.081918 3707 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
I20260810 23:02:13.081983 3707 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6407f8fc83b4f10a8cbac7c1410d465; no voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476
I20260810 23:02:13.082069 4407 raft_consensus.cc:2726] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:02:13.236006 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:13.235778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:59917)
0810 23:02:13.235915 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:13.235960 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260810 23:02:13.306699 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:13.306501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:34851)
0810 23:02:13.306622 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:13.306663 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260810 23:02:13.438835 4412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:13.438634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:45051)
0810 23:02:13.438763 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:13.438795 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260810 23:02:14.236716 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:14.236486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:55043)
0810 23:02:14.236632 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:14.236668 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260810 23:02:14.307327 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:14.307122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:45301)
0810 23:02:14.307246 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:14.307286 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260810 23:02:14.439424 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:14.439221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:52663)
0810 23:02:14.439352 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:14.439385 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260810 23:02:15.237335 4422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:15.237121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:54605)
0810 23:02:15.237257 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:15.237288 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260810 23:02:15.307983 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:15.307783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:52397)
0810 23:02:15.307908 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:15.307941 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260810 23:02:15.440296 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:15.440061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:37013)
0810 23:02:15.440217 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:15.440255 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260810 23:02:15.589704 4425 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:15.589810 4425 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:15.590000 4425 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
W20260810 23:02:15.590435 3707 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
W20260810 23:02:15.590610 3706 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
I20260810 23:02:15.590708 3706 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6407f8fc83b4f10a8cbac7c1410d465; no voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476
I20260810 23:02:15.590830 4425 raft_consensus.cc:2726] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:02:16.238004 4427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:16.237787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:53539)
0810 23:02:16.237921 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:16.237957 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260810 23:02:16.308645 4430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:16.308383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:39409)
0810 23:02:16.308562 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:16.308600 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260810 23:02:16.441020 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:16.440824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:54571)
0810 23:02:16.440945 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:16.440982 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260810 23:02:17.238763 4435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:17.238542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:33405)
0810 23:02:17.238681 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:17.238717 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260810 23:02:17.309278 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:17.309056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:44877)
0810 23:02:17.309186 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:17.309224 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260810 23:02:17.441759 4441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:17.441511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:46645)
0810 23:02:17.441663 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:17.441708 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260810 23:02:18.239557 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:18.239307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:53261)
0810 23:02:18.239460 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:18.239505 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260810 23:02:18.309952 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:18.309741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:34953)
0810 23:02:18.309873 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:18.309909 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260810 23:02:18.442654 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:18.442333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:58407)
0810 23:02:18.442563 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:18.442608 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":63,"threads_started":1}
I20260810 23:02:19.240433 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:19.240158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:60859)
0810 23:02:19.240347 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:19.240386 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":114,"threads_started":1}
I20260810 23:02:19.310611 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:19.310387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:43429)
0810 23:02:19.310525 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:19.310562 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260810 23:02:19.443367 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:19.443164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:60459)
0810 23:02:19.443292 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:19.443325 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260810 23:02:20.241225 4452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:20.240971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:52461)
0810 23:02:20.241129 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:20.241161 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260810 23:02:20.311398 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:20.311173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:59763)
0810 23:02:20.311318 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:20.311356 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260810 23:02:20.444160 4458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:20.443905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:55179)
0810 23:02:20.444072 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:20.444114 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260810 23:02:21.241943 4459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:21.241677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:46575)
0810 23:02:21.241834 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:21.241872 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260810 23:02:21.312103 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:21.311867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:43593)
0810 23:02:21.312024 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:21.312059 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260810 23:02:21.444914 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:21.444702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:37835)
0810 23:02:21.444838 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:21.444874 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260810 23:02:21.605015 4462 raft_consensus.cc:482] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:21.605171 4462 raft_consensus.cc:504] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bef7fa8d2f143f8b580a4d41bd4c89a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 43877 } } peers { permanent_uuid: "b2c9fcf9046a4389b9cb2db7af69a476" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 36083 } } peers { permanent_uuid: "f6407f8fc83b4f10a8cbac7c1410d465" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 40291 } }
I20260810 23:02:21.605438 4462 leader_election.cc:288] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877), b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083)
W20260810 23:02:21.605798 3706 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bef7fa8d2f143f8b580a4d41bd4c89a (127.6.47.131:43877): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
W20260810 23:02:21.605890 3707 leader_election.cc:341] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b2c9fcf9046a4389b9cb2db7af69a476 (127.6.47.130:36083): Not found: Tablet not found: 760922349aab4eb0bc62c7ed705854ab
I20260810 23:02:21.605981 3707 leader_election.cc:302] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6407f8fc83b4f10a8cbac7c1410d465; no voters: 0bef7fa8d2f143f8b580a4d41bd4c89a, b2c9fcf9046a4389b9cb2db7af69a476
I20260810 23:02:21.606076 4462 raft_consensus.cc:2726] T 760922349aab4eb0bc62c7ed705854ab P f6407f8fc83b4f10a8cbac7c1410d465 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 23:02:22.242764 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:22.242413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:51271)
0810 23:02:22.242678 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:22.242716 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":67,"threads_started":1}
I20260810 23:02:22.312925 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:22.312714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:37721)
0810 23:02:22.312845 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:22.312880 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":66,"threads_started":1}
I20260810 23:02:22.445570 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:22.445360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:49279)
0810 23:02:22.445477 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:22.445515 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260810 23:02:23.243834 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:23.243601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.129:47313)
0810 23:02:23.243759 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:23.243789 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260810 23:02:23.313663 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:23.313440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.130:43755)
0810 23:02:23.313573 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:23.313622 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260810 23:02:23.442461 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 3609
I20260810 23:02:23.446492 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:23.446031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38981 (local address 127.6.47.131:37327)
0810 23:02:23.446410 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:23.446452 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.190:38981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":56,"threads_started":1}
I20260810 23:02:23.451213 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 3922
I20260810 23:02:23.467880 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 4156
2026-08-10T23:02:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58553 ms)
[----------] 30 tests from SecurityITest (241772 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:23 dist-test-slave-rwrg krb5kdc[4482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:23 dist-test-slave-rwrg krb5kdc[4482](info): set up 2 sockets
Aug 10 23:02:23 dist-test-slave-rwrg krb5kdc[4482](info): commencing operation
krb5kdc: starting...
W20260810 23:02:24.778170 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:24 dist-test-slave-rwrg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:24.899389 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40013
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:24.981058 4509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:24.981263 4509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:24.981284 4509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:24.982419 4509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:24.982458 4509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:24.982471 4509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:24.982537 4509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:24.982563 4509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:24.984131 4509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:24.984380 4509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:24.984622 4509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:24.989372 4519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:24.989388 4522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:24.989437 4520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:24.989727 4509 server_base.cc:1034] running on GCE node
I20260810 23:02:24.989965 4509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:24.990193 4509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:24.991355 4509 hybrid_clock.cc:648] HybridClock initialized: now 1786402944991336 us; error 31 us; skew 500 ppm
Aug 10 23:02:24 dist-test-slave-rwrg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402944, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:24.993892 4509 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:24.994342 4509 webserver.cc:460] Webserver started at http://127.6.47.190:42437/ using document root <none> and password file <none>
I20260810 23:02:24.994537 4509 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:24.994578 4509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:24.994668 4509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:24.995628 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "d8575dfb2c50480fbc4031bb9f314115"
format_stamp: "Formatted at 2026-08-10 23:02:24 on dist-test-slave-rwrg"
server_key: "d5dd14b98f2811809062c58e614aefca"
server_key_iv: "3c87ded0bb1012eed5c226af662c55b8"
server_key_version: "encryptionkey@0"
I20260810 23:02:24.995927 4509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "d8575dfb2c50480fbc4031bb9f314115"
format_stamp: "Formatted at 2026-08-10 23:02:24 on dist-test-slave-rwrg"
server_key: "d5dd14b98f2811809062c58e614aefca"
server_key_iv: "3c87ded0bb1012eed5c226af662c55b8"
server_key_version: "encryptionkey@0"
I20260810 23:02:24.997293 4509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:24.997975 4538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:24.998174 4509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:24.998270 4509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "d8575dfb2c50480fbc4031bb9f314115"
format_stamp: "Formatted at 2026-08-10 23:02:24 on dist-test-slave-rwrg"
server_key: "d5dd14b98f2811809062c58e614aefca"
server_key_iv: "3c87ded0bb1012eed5c226af662c55b8"
server_key_version: "encryptionkey@0"
I20260810 23:02:24.998325 4509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:25.010831 4509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:25.012801 4509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:25.012916 4509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:25.016799 4509 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:40013
I20260810 23:02:25.016862 4639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:40013 every 8 connection(s)
I20260810 23:02:25.017227 4509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:25.018146 4641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:25.020414 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: Bootstrap starting.
I20260810 23:02:25.021040 4641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:25.021373 4641 log.cc:824] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:25.022130 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: No bootstrap required, opened a new log
I20260810 23:02:25.023847 4641 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } }
I20260810 23:02:25.024008 4641 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:25.024055 4641 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8575dfb2c50480fbc4031bb9f314115, State: Initialized, Role: FOLLOWER
I20260810 23:02:25.024246 4641 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } }
I20260810 23:02:25.024348 4641 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:25.024394 4641 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:25.024453 4641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:25.024619 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 4509
I20260810 23:02:25.024761 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:25.024955 6334 external_mini_cluster.cc:1428] Setting key fff73e93a5023baaba48efa44b60c5e0
I20260810 23:02:25.025235 4641 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } }
I20260810 23:02:25.025357 4641 leader_election.cc:302] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8575dfb2c50480fbc4031bb9f314115; no voters:
I20260810 23:02:25.025538 4641 leader_election.cc:288] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:25.025599 4652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:25.025748 4652 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [term 1 LEADER]: Becoming Leader. State: Replica: d8575dfb2c50480fbc4031bb9f314115, State: Running, Role: LEADER
I20260810 23:02:25.025811 4641 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:25.025880 4652 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } }
I20260810 23:02:25.026222 4652 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8575dfb2c50480fbc4031bb9f314115. Latest consensus state: current_term: 1 leader_uuid: "d8575dfb2c50480fbc4031bb9f314115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } } }
I20260810 23:02:25.026314 4652 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:25.026549 4672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:25.026505 4655 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8575dfb2c50480fbc4031bb9f314115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8575dfb2c50480fbc4031bb9f314115" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40013 } } }
I20260810 23:02:25.027441 4655 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:25.027541 4672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:25.030212 4672 catalog_manager.cc:1347] Generated new cluster ID: 4fac01588d164e9598f6fe417be45e4f
I20260810 23:02:25.030269 4672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:25.036185 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.025438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.6.47.190:40013)
0810 23:02:25.025625 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.025636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.025688 (+ 52us) server_negotiation.cc:407] Connection header received
0810 23:02:25.025977 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.025985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.026042 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.026181 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:25.027170 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.028633 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.029714 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.030282 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.030408 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.032914 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:25.032919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:25.032919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:25.032937 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:25.034636 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.035012 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.035016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.035016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.035131 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.035358 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.035359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.035359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.035597 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:25.035766 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.035874 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.036016 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260810 23:02:25.057758 4672 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:25.058336 4672 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:25.069295 4672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: Generated new TSK 0
I20260810 23:02:25.069538 4672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:02:25.099452 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:25.182961 4697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:25.183166 4697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:25.183188 4697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:25.184405 4697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:25.184448 4697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:25.184473 4697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:25.186009 4697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:25.186260 4697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:25.186532 4697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:25.191624 4705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:25.191637 4710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.191689 4697 server_base.cc:1034] running on GCE node
W20260810 23:02:25.191638 4704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.192024 4697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:25.192276 4697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:25.193470 4697 hybrid_clock.cc:648] HybridClock initialized: now 1786402945193468 us; error 59 us; skew 500 ppm
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:25.196079 4697 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:02:25.196532 4697 webserver.cc:460] Webserver started at http://127.6.47.129:35319/ using document root <none> and password file <none>
I20260810 23:02:25.196835 4697 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:25.196910 4697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:25.197058 4697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:25.198083 4697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "7cb6350a334341be855829ea4ea76c43"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "b3a4ebb79d6d2ec1706b7c66290925cc"
server_key_iv: "d687f70cc37e9983ac9989cb8bc1b641"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.198455 4697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "7cb6350a334341be855829ea4ea76c43"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "b3a4ebb79d6d2ec1706b7c66290925cc"
server_key_iv: "d687f70cc37e9983ac9989cb8bc1b641"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.199795 4697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:25.200572 4723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.200757 4697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:25.200929 4697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "7cb6350a334341be855829ea4ea76c43"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "b3a4ebb79d6d2ec1706b7c66290925cc"
server_key_iv: "d687f70cc37e9983ac9989cb8bc1b641"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.201014 4697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:25.209968 4697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:25.212580 4697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:25.212745 4697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:25.212972 4697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:25.213308 4697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:25.213376 4697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.213436 4697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:25.213466 4697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.220120 4697 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:38937
I20260810 23:02:25.220160 4930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:38937 every 8 connection(s)
I20260810 23:02:25.220481 4697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:25.224843 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 4697
I20260810 23:02:25.224941 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:25.225142 6334 external_mini_cluster.cc:1428] Setting key 998ec19db74704eb5a41564c03230fe6
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:25.230908 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.221374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:52425 (local address 127.6.47.190:40013)
0810 23:02:25.221461 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.221464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.221649 (+ 185us) server_negotiation.cc:407] Connection header received
0810 23:02:25.222173 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.222175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.222192 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.222239 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:25.223164 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.223660 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.224654 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.224957 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.225030 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.227451 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:25.227452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:25.227452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:25.227459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:25.229419 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.229948 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.229949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.229949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.230000 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.230306 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.230307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.230307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.230403 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:25.230452 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.230779 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.230845 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:02:25.231329 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.221339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40013 (local address 127.6.47.129:52425)
0810 23:02:25.221523 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.221565 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.222051 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.222300 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.222304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.222871 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:25.223047 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.223052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.223759 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.223761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.224556 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.224559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.225054 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.225055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.225164 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.225785 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:25.225795 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:25.227353 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:25.229527 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.229536 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.229538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.229871 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.230067 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.230068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.230068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.230238 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.230468 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:25.230475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:25.230639 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.230756 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.230943 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260810 23:02:25.231918 4931 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40013
I20260810 23:02:25.232033 4931 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:25.232204 4931 heartbeater.cc:511] Master 127.6.47.190:40013 requested a full tablet report, sending...
I20260810 23:02:25.232748 4577 ts_manager.cc:194] Registered new tserver with Master: 7cb6350a334341be855829ea4ea76c43 (127.6.47.129:38937)
I20260810 23:02:25.233362 4577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:52425
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:02:25.253247 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:25.334869 4939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:25.335126 4939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:25.335166 4939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:25.336293 4939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:25.336364 4939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:25.336411 4939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:25.337955 4939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:25.338243 4939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:25.338498 4939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:25.343822 4947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.343982 4939 server_base.cc:1034] running on GCE node
W20260810 23:02:25.343943 4950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:25.343878 4954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.344297 4939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:25.344560 4939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:25.345716 4939 hybrid_clock.cc:648] HybridClock initialized: now 1786402945345711 us; error 33 us; skew 500 ppm
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:25.348299 4939 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:02:25.348770 4939 webserver.cc:460] Webserver started at http://127.6.47.130:42693/ using document root <none> and password file <none>
I20260810 23:02:25.349009 4939 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:25.349082 4939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:25.349267 4939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:25.350338 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "96a3805d19394cf7a63b58199d0795c9"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "797dedec08182cf95921d5438bf6f488"
server_key_iv: "9d95ab32b3cb3539ae2fc2a276ee6568"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.350723 4939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "96a3805d19394cf7a63b58199d0795c9"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "797dedec08182cf95921d5438bf6f488"
server_key_iv: "9d95ab32b3cb3539ae2fc2a276ee6568"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.352149 4939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:02:25.352948 4963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.353163 4939 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:02:25.353276 4939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "96a3805d19394cf7a63b58199d0795c9"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "797dedec08182cf95921d5438bf6f488"
server_key_iv: "9d95ab32b3cb3539ae2fc2a276ee6568"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.353353 4939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:25.380776 4939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:25.382860 4939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:25.383037 4939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:25.383294 4939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:25.383639 4939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:25.383697 4939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.383736 4939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:25.383764 4939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.390156 4939 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:38713
I20260810 23:02:25.390208 5178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:38713 every 8 connection(s)
I20260810 23:02:25.390575 4939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:25.398355 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 4939
I20260810 23:02:25.398484 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:25.398710 6334 external_mini_cluster.cc:1428] Setting key 5357c7c6223206d3730bff69a1dcdea2
I20260810 23:02:25.400642 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.391351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:45173 (local address 127.6.47.190:40013)
0810 23:02:25.391395 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.391398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.391594 (+ 196us) server_negotiation.cc:407] Connection header received
0810 23:02:25.392160 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.392162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.392175 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.392222 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:25.393304 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.393803 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.394865 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.395066 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.395162 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.397491 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:25.397493 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:25.397494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:25.397501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:25.399111 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.399555 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.399556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.399557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.399641 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.399876 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.399877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.399877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.400023 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:25.400091 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.400364 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.400476 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:02:25.400751 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.391310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40013 (local address 127.6.47.130:45173)
0810 23:02:25.391460 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.391507 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.392033 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.392306 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.392310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.392988 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:25.393190 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.393195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.393908 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.393909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.394753 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.394756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.395175 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.395176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.395304 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.395822 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:25.395831 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:25.397349 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:25.399194 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.399201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.399203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.399499 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.399710 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.399710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.399711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.399811 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.400089 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:25.400093 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:25.400206 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.400288 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.400479 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260810 23:02:25.401330 5179 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40013
I20260810 23:02:25.401468 5179 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:25.401679 5179 heartbeater.cc:511] Master 127.6.47.190:40013 requested a full tablet report, sending...
I20260810 23:02:25.402073 4577 ts_manager.cc:194] Registered new tserver with Master: 96a3805d19394cf7a63b58199d0795c9 (127.6.47.130:38713)
I20260810 23:02:25.402463 4577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:45173
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:02:25.425354 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:25.529170 5186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:25.529417 5186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:25.529457 5186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:25.530609 5186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:25.530683 5186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:25.530728 5186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:25.532262 5186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40013
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:25.532575 5186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:25.532858 5186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:25.537930 5196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.537956 5186 server_base.cc:1034] running on GCE node
W20260810 23:02:25.537940 5194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:25.537940 5192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.538403 5186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:25.538655 5186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:25.539829 5186 hybrid_clock.cc:648] HybridClock initialized: now 1786402945539817 us; error 24 us; skew 500 ppm
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:25.542590 5186 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:02:25.543129 5186 webserver.cc:460] Webserver started at http://127.6.47.131:42113/ using document root <none> and password file <none>
I20260810 23:02:25.543388 5186 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:25.543433 5186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:25.543574 5186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:25.544663 5186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "8645a9f44d094c2f9b10237ec41c6988"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "23c26fbc2ca618e0aeaaa600587e06b9"
server_key_iv: "4474496fb932850c775236cf87d10622"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.545046 5186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "8645a9f44d094c2f9b10237ec41c6988"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "23c26fbc2ca618e0aeaaa600587e06b9"
server_key_iv: "4474496fb932850c775236cf87d10622"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.546418 5186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:02:25.547181 5212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.547361 5186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:25.547472 5186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "8645a9f44d094c2f9b10237ec41c6988"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "23c26fbc2ca618e0aeaaa600587e06b9"
server_key_iv: "4474496fb932850c775236cf87d10622"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.547547 5186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:25.570784 5186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:25.573153 5186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:25.573318 5186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:25.573547 5186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:25.573869 5186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:25.573931 5186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.573972 5186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:25.574000 5186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.580438 5186 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:40231
I20260810 23:02:25.580485 5415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:40231 every 8 connection(s)
I20260810 23:02:25.580860 5186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:25.586251 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 5186
I20260810 23:02:25.586354 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:25.586563 6334 external_mini_cluster.cc:1428] Setting key 09e84596068c32ca84808c2a72542c93
Aug 10 23:02:25 dist-test-slave-rwrg krb5kdc[4482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402945, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:25.590292 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.581594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:56885 (local address 127.6.47.190:40013)
0810 23:02:25.581663 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.581666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.581888 (+ 222us) server_negotiation.cc:407] Connection header received
0810 23:02:25.582380 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.582382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.582392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.582462 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:25.583446 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.583941 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.584978 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.585179 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.585235 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.587714 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:25.587715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:25.587716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:25.587723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:25.589119 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.589505 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.589506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.589507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.589558 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.589780 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.589780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.589781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.589883 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:25.589934 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.590165 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.590228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:02:25.590480 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.581578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40013 (local address 127.6.47.131:56885)
0810 23:02:25.581782 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.581820 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.582295 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.582543 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.582546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.583130 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:25.583331 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.583336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.584027 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.584028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.584903 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.584906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.585257 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.585258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.585361 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.585829 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:25.585837 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:25.587600 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:25.589197 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.589203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.589206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.589438 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.589617 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:25.589618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:25.589618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:25.589724 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:25.589938 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:25.589943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:25.590053 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.590134 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.590264 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260810 23:02:25.591078 5416 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40013
I20260810 23:02:25.591163 5416 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:25.591311 5416 heartbeater.cc:511] Master 127.6.47.190:40013 requested a full tablet report, sending...
I20260810 23:02:25.591634 4577 ts_manager.cc:194] Registered new tserver with Master: 8645a9f44d094c2f9b10237ec41c6988 (127.6.47.131:40231)
I20260810 23:02:25.592005 4577 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:56885
I20260810 23:02:25.598446 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:25.605440 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.599800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.6.47.190:40013)
0810 23:02:25.599842 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.599844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.599926 (+ 82us) server_negotiation.cc:407] Connection header received
0810 23:02:25.599966 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.599966 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.599974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.600011 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:25.600614 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.601153 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.601869 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.602087 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.602140 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.603224 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:25.603226 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:25.603226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:25.603232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:25.604459 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.604818 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.604819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.604819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.604872 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:25.605033 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:25.605033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:25.605033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:25.605158 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:25.605231 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.605322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.605380 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:02:25.606966 4577 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:25.608542 4577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:25.615099 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.610805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:38937 (local address 127.0.0.1:40488)
0810 23:02:25.611150 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.611198 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.611267 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.611564 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.611565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.611567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:25.611595 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.611596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.612755 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.612756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.614070 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.614073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.614751 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.614832 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.614844 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.614967 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.615024 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260810 23:02:25.615113 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.610943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40488 (local address 127.6.47.129:38937)
0810 23:02:25.611149 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.611153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.611351 (+ 198us) server_negotiation.cc:407] Connection header received
0810 23:02:25.611398 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.611400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.611415 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.611483 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:25.611671 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.612642 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614176 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.614654 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614749 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.614812 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.614987 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.615025 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260810 23:02:25.615242 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.610805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:38713 (local address 127.0.0.1:43308)
0810 23:02:25.611150 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.611196 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.611261 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.611513 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.611516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.611518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:25.611579 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.611582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.613048 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.613049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.614106 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.614109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.615020 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.615022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.615087 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.615092 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.615122 (+ 30us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.615166 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":6,"thread_start_us":76,"threads_started":1}
I20260810 23:02:25.615429 5448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.611097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.6.47.131:40231)
0810 23:02:25.611410 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.611416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.611426 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:25.611482 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.611484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.611499 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.611566 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:25.611814 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.612896 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614183 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.614913 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614988 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.615044 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.615292 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.615338 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":108,"threads_started":1}
I20260810 23:02:25.615686 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.610942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43308 (local address 127.6.47.130:38713)
0810 23:02:25.611232 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.611236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.611247 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:02:25.611370 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.611373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.611390 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.611474 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:25.611665 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.612885 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.614215 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.614912 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.615476 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.615548 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.615573 (+ 25us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.615614 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":90,"threads_started":1}
I20260810 23:02:25.615839 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.611001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:40231 (local address 127.0.0.1:55488)
0810 23:02:25.611296 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.611305 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.611348 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.611665 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.611666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.611668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:25.611727 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.611729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.613021 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.613022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.614092 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.614094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.615074 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.615075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.615132 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.615135 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.615194 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.615227 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260810 23:02:25.616058 4807 tablet_service.cc:1467] Processing CreateTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 (DEFAULT_TABLE table=test-table [id=56dc84c59e314263bb78ce644f4f2507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:25.616446 4807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ac4dd9e8ac427996eae0b6ad6d2631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:25.616374 5067 tablet_service.cc:1467] Processing CreateTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 (DEFAULT_TABLE table=test-table [id=56dc84c59e314263bb78ce644f4f2507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:25.616575 5301 tablet_service.cc:1467] Processing CreateTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 (DEFAULT_TABLE table=test-table [id=56dc84c59e314263bb78ce644f4f2507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:25.616735 5067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ac4dd9e8ac427996eae0b6ad6d2631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:25.616878 5301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ac4dd9e8ac427996eae0b6ad6d2631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:25.619230 5449 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Bootstrap starting.
I20260810 23:02:25.619951 5449 tablet_bootstrap.cc:654] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:25.620011 5452 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Bootstrap starting.
I20260810 23:02:25.620393 5454 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Bootstrap starting.
I20260810 23:02:25.620632 5449 log.cc:824] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:25.620766 5452 tablet_bootstrap.cc:654] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:25.621101 5454 tablet_bootstrap.cc:654] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:25.621323 5452 log.cc:824] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:25.621507 5449 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: No bootstrap required, opened a new log
I20260810 23:02:25.621585 5449 ts_tablet_manager.cc:1397] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:25.621727 5454 log.cc:824] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:25.622128 5452 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: No bootstrap required, opened a new log
I20260810 23:02:25.622181 5452 ts_tablet_manager.cc:1397] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:25.622880 5454 tablet_bootstrap.cc:492] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: No bootstrap required, opened a new log
I20260810 23:02:25.622953 5454 ts_tablet_manager.cc:1397] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:02:25.623463 5449 raft_consensus.cc:348] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.623627 5449 raft_consensus.cc:374] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:25.623682 5449 raft_consensus.cc:729] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a3805d19394cf7a63b58199d0795c9, State: Initialized, Role: FOLLOWER
I20260810 23:02:25.623803 5449 consensus_queue.cc:260] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.624109 5179 heartbeater.cc:503] Master 127.6.47.190:40013 was elected leader, sending a full tablet report...
I20260810 23:02:25.624123 5449 ts_tablet_manager.cc:1428] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 23:02:25.624193 5452 raft_consensus.cc:348] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.624369 5452 raft_consensus.cc:374] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:25.624413 5452 raft_consensus.cc:729] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb6350a334341be855829ea4ea76c43, State: Initialized, Role: FOLLOWER
I20260810 23:02:25.624562 5452 consensus_queue.cc:260] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.624861 5452 ts_tablet_manager.cc:1428] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 23:02:25.624930 4931 heartbeater.cc:503] Master 127.6.47.190:40013 was elected leader, sending a full tablet report...
I20260810 23:02:25.625082 5454 raft_consensus.cc:348] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.625285 5454 raft_consensus.cc:374] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:25.625339 5454 raft_consensus.cc:729] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8645a9f44d094c2f9b10237ec41c6988, State: Initialized, Role: FOLLOWER
I20260810 23:02:25.625484 5454 consensus_queue.cc:260] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.625831 5416 heartbeater.cc:503] Master 127.6.47.190:40013 was elected leader, sending a full tablet report...
I20260810 23:02:25.625829 5454 ts_tablet_manager.cc:1428] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260810 23:02:25.641121 5180 tablet.cc:2369] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:02:25.721381 4932 tablet.cc:2369] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:02:25.750425 5461 raft_consensus.cc:482] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:25.750563 5461 raft_consensus.cc:504] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.750950 5461 leader_election.cc:288] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cb6350a334341be855829ea4ea76c43 (127.6.47.129:38937), 8645a9f44d094c2f9b10237ec41c6988 (127.6.47.131:40231)
I20260810 23:02:25.753947 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.751258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:40503 (local address 127.6.47.129:38937)
0810 23:02:25.751326 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.751328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.751342 (+ 14us) server_negotiation.cc:407] Connection header received
0810 23:02:25.751379 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.751380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.751392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.751472 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:25.751685 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.752153 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753201 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.753574 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753660 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.753678 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.753840 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.753876 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 23:02:25.753948 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.751191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:38937 (local address 127.6.47.130:40503)
0810 23:02:25.751241 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.751253 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.751304 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.751514 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.751516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.751518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:25.751594 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.751596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.752248 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.752249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.753092 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.753094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.753657 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.753754 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.753758 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.753834 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.753875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":7}
I20260810 23:02:25.754029 5448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.751454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:45009 (local address 127.6.47.131:40231)
0810 23:02:25.751509 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.751512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.751525 (+ 13us) server_negotiation.cc:407] Connection header received
0810 23:02:25.751613 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.751614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.751627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.751703 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:25.751863 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.752330 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753261 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.753660 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753749 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.753758 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.753912 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.753949 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:02:25.754281 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.751311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:40231 (local address 127.6.47.130:45009)
0810 23:02:25.751479 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.751490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:25.751538 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:25.751708 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:25.751709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:25.751711 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:25.751789 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.751791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.752420 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.752421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.753168 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:25.753170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.753751 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.753752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:25.753831 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.753834 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:25.753906 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 23:02:25.753946 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260810 23:02:25.754455 4842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ac4dd9e8ac427996eae0b6ad6d2631" candidate_uuid: "96a3805d19394cf7a63b58199d0795c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb6350a334341be855829ea4ea76c43" is_pre_election: true
I20260810 23:02:25.754640 4842 raft_consensus.cc:2454] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a3805d19394cf7a63b58199d0795c9 in term 0.
I20260810 23:02:25.754671 5338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ac4dd9e8ac427996eae0b6ad6d2631" candidate_uuid: "96a3805d19394cf7a63b58199d0795c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8645a9f44d094c2f9b10237ec41c6988" is_pre_election: true
I20260810 23:02:25.754846 5338 raft_consensus.cc:2454] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96a3805d19394cf7a63b58199d0795c9 in term 0.
I20260810 23:02:25.754948 4969 leader_election.cc:302] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cb6350a334341be855829ea4ea76c43, 96a3805d19394cf7a63b58199d0795c9; no voters:
I20260810 23:02:25.755101 5461 raft_consensus.cc:2781] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:25.755189 5461 raft_consensus.cc:482] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:25.755223 5461 raft_consensus.cc:3037] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:25.755991 5461 raft_consensus.cc:504] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.756138 5461 leader_election.cc:288] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [CANDIDATE]: Term 1 election: Requested vote from peers 7cb6350a334341be855829ea4ea76c43 (127.6.47.129:38937), 8645a9f44d094c2f9b10237ec41c6988 (127.6.47.131:40231)
I20260810 23:02:25.756349 5338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ac4dd9e8ac427996eae0b6ad6d2631" candidate_uuid: "96a3805d19394cf7a63b58199d0795c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8645a9f44d094c2f9b10237ec41c6988"
I20260810 23:02:25.756341 4842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ac4dd9e8ac427996eae0b6ad6d2631" candidate_uuid: "96a3805d19394cf7a63b58199d0795c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cb6350a334341be855829ea4ea76c43"
I20260810 23:02:25.756453 5338 raft_consensus.cc:3037] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:25.756453 4842 raft_consensus.cc:3037] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:25.757294 4842 raft_consensus.cc:2454] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a3805d19394cf7a63b58199d0795c9 in term 1.
I20260810 23:02:25.757318 5338 raft_consensus.cc:2454] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96a3805d19394cf7a63b58199d0795c9 in term 1.
I20260810 23:02:25.757494 4969 leader_election.cc:302] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8645a9f44d094c2f9b10237ec41c6988, 96a3805d19394cf7a63b58199d0795c9; no voters:
I20260810 23:02:25.757606 5461 raft_consensus.cc:2781] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:25.757767 5461 raft_consensus.cc:686] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 1 LEADER]: Becoming Leader. State: Replica: 96a3805d19394cf7a63b58199d0795c9, State: Running, Role: LEADER
I20260810 23:02:25.757863 5461 consensus_queue.cc:237] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } }
I20260810 23:02:25.758737 4574 catalog_manager.cc:5495] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96a3805d19394cf7a63b58199d0795c9 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "96a3805d19394cf7a63b58199d0795c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96a3805d19394cf7a63b58199d0795c9" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 } health_report { overall_health: UNKNOWN } } }
I20260810 23:02:25.799363 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.796929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43310 (local address 127.6.47.130:38713)
0810 23:02:25.797007 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:25.797008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:25.797017 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:02:25.797078 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:25.797079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:25.797101 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:25.797165 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:25.797277 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.797747 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.798517 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:25.798739 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:25.798789 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:25.798955 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:25.799092 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:25.799167 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:25.799211 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 23:02:25.799229 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:02:25.801450 5338 raft_consensus.cc:1261] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 1 FOLLOWER]: Refusing update from remote peer 96a3805d19394cf7a63b58199d0795c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:25.801466 4842 raft_consensus.cc:1261] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 7cb6350a334341be855829ea4ea76c43 [term 1 FOLLOWER]: Refusing update from remote peer 96a3805d19394cf7a63b58199d0795c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:25.801968 5472 consensus_queue.cc:1035] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8645a9f44d094c2f9b10237ec41c6988" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 40231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:25.802143 5461 consensus_queue.cc:1035] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cb6350a334341be855829ea4ea76c43" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 38937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:25.805277 5481 mvcc.cc:205] Tried to move back new op lower bound from 7317106465999642624 to 7317106465824776192. Current Snapshot: MvccSnapshot[applied={T|T < 7317106465999642624}]
I20260810 23:02:25.805792 5480 mvcc.cc:205] Tried to move back new op lower bound from 7317106465999642624 to 7317106465824776192. Current Snapshot: MvccSnapshot[applied={T|T < 7317106465999642624}]
I20260810 23:02:25.806943 4574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:25.807082 4574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:25.807883 4574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8575dfb2c50480fbc4031bb9f314115: Sending DeleteTablet for 3 replicas of tablet e1ac4dd9e8ac427996eae0b6ad6d2631
I20260810 23:02:25.808215 5067 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:25 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:43308
I20260810 23:02:25.808270 4807 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:25 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:40488
I20260810 23:02:25.808322 5301 tablet_service.cc:1514] Processing DeleteTablet for tablet e1ac4dd9e8ac427996eae0b6ad6d2631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:25 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:55488
I20260810 23:02:25.808425 5497 tablet_replica.cc:330] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: stopping tablet replica
I20260810 23:02:25.808535 5497 raft_consensus.cc:2229] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:25.808635 5499 tablet_replica.cc:330] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: stopping tablet replica
I20260810 23:02:25.808650 5497 raft_consensus.cc:2258] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:25.808753 5499 raft_consensus.cc:2229] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:25.808866 5499 raft_consensus.cc:2258] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:25.808996 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 4697
I20260810 23:02:25.809095 5497 ts_tablet_manager.cc:1905] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:25.813710 5497 ts_tablet_manager.cc:1918] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:25.813795 5497 log.cc:1196] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/e1ac4dd9e8ac427996eae0b6ad6d2631
I20260810 23:02:25.814131 5497 ts_tablet_manager.cc:1939] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 96a3805d19394cf7a63b58199d0795c9: Deleting consensus metadata
I20260810 23:02:25.816025 5499 ts_tablet_manager.cc:1905] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:25.818037 5499 ts_tablet_manager.cc:1918] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:25.818110 5499 log.cc:1196] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/e1ac4dd9e8ac427996eae0b6ad6d2631
I20260810 23:02:25.818351 5499 ts_tablet_manager.cc:1939] T e1ac4dd9e8ac427996eae0b6ad6d2631 P 8645a9f44d094c2f9b10237ec41c6988: Deleting consensus metadata
I20260810 23:02:25.821646 4543 catalog_manager.cc:4841] TS 96a3805d19394cf7a63b58199d0795c9 (127.6.47.130:38713): tablet e1ac4dd9e8ac427996eae0b6ad6d2631 (table test-table [id=56dc84c59e314263bb78ce644f4f2507]) successfully deleted
I20260810 23:02:25.823508 4544 catalog_manager.cc:4841] TS 8645a9f44d094c2f9b10237ec41c6988 (127.6.47.131:40231): tablet e1ac4dd9e8ac427996eae0b6ad6d2631 (table test-table [id=56dc84c59e314263bb78ce644f4f2507]) successfully deleted
W20260810 23:02:25.827879 4544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:38937 (error 108)
I20260810 23:02:25.828128 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 4939
I20260810 23:02:25.837019 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:25.836842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:38937 (local address 127.0.0.1:40496)
0810 23:02:25.836905 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:25.836960 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:38937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 23:02:25.837240 4544 catalog_manager.cc:4584] TS 7cb6350a334341be855829ea4ea76c43 (127.6.47.129:38937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1ac4dd9e8ac427996eae0b6ad6d2631: Network error: Client connection negotiation failed: client connection to 127.6.47.129:38937: connect: Connection refused (error 111)
I20260810 23:02:25.841329 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 5186
I20260810 23:02:25.852715 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 4509
2026-08-10T23:02:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2384 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T23:02:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:25Z Disabled control of system clock
I20260810 23:02:25.890169 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35175
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:35175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:25.971318 5508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:25.971549 5508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:25.971567 5508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:25.972689 5508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:25.972724 5508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:25.972738 5508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:25.972750 5508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:25.972762 5508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:25.974040 5508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:35175
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:25.974285 5508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:25.974535 5508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:25.979427 5521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:25.979410 5525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:25.979415 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:25.979691 5508 server_base.cc:1034] running on GCE node
I20260810 23:02:25.979918 5508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:25.980167 5508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:25.981303 5508 hybrid_clock.cc:648] HybridClock initialized: now 1786402945981277 us; error 64 us; skew 500 ppm
I20260810 23:02:25.982650 5508 webserver.cc:460] Webserver started at http://127.6.47.190:36881/ using document root <none> and password file <none>
I20260810 23:02:25.982856 5508 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:25.982898 5508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:25.982987 5508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:25.984033 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "683af1f7534d45dabd4b8ea1dc00c3ea"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "6ca97b6bc365aa5bcb2c225616fdc810"
server_key_iv: "3880ca23513dfd20349b42e7fc48c8cc"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.984356 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "683af1f7534d45dabd4b8ea1dc00c3ea"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "6ca97b6bc365aa5bcb2c225616fdc810"
server_key_iv: "3880ca23513dfd20349b42e7fc48c8cc"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.985697 5508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:25.986378 5536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:25.986567 5508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:25.986652 5508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "683af1f7534d45dabd4b8ea1dc00c3ea"
format_stamp: "Formatted at 2026-08-10 23:02:25 on dist-test-slave-rwrg"
server_key: "6ca97b6bc365aa5bcb2c225616fdc810"
server_key_iv: "3880ca23513dfd20349b42e7fc48c8cc"
server_key_version: "encryptionkey@0"
I20260810 23:02:25.986706 5508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:26.020413 5508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:26.020810 5508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:26.020931 5508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:26.025126 5508 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:35175
I20260810 23:02:26.025193 5628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:35175 every 8 connection(s)
I20260810 23:02:26.025570 5508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:26.026737 5629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:26.029291 5629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: Bootstrap starting.
I20260810 23:02:26.029855 5629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:26.030107 5629 log.cc:824] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:26.030791 5629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: No bootstrap required, opened a new log
I20260810 23:02:26.032387 5629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } }
I20260810 23:02:26.032603 5629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:26.032631 5629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683af1f7534d45dabd4b8ea1dc00c3ea, State: Initialized, Role: FOLLOWER
I20260810 23:02:26.032745 5629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } }
I20260810 23:02:26.032801 5629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:26.032823 5629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:26.032856 5629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:26.033354 5629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } }
I20260810 23:02:26.033443 5629 leader_election.cc:302] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 683af1f7534d45dabd4b8ea1dc00c3ea; no voters:
I20260810 23:02:26.033583 5629 leader_election.cc:288] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:26.033676 5633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:26.033874 5633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [term 1 LEADER]: Becoming Leader. State: Replica: 683af1f7534d45dabd4b8ea1dc00c3ea, State: Running, Role: LEADER
I20260810 23:02:26.033975 5629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:26.033977 5633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } }
I20260810 23:02:26.034492 5634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } } }
I20260810 23:02:26.034588 5635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 683af1f7534d45dabd4b8ea1dc00c3ea. Latest consensus state: current_term: 1 leader_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "683af1f7534d45dabd4b8ea1dc00c3ea" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35175 } } }
I20260810 23:02:26.034588 5634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:26.034734 5635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:26.034932 5646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:26.035171 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 5508
I20260810 23:02:26.035302 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:26.035490 5646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:26.035521 6334 external_mini_cluster.cc:1428] Setting key 46835141e94f8071e106087c3cd7e23a
I20260810 23:02:26.037554 5646 catalog_manager.cc:1347] Generated new cluster ID: 2c725759126e4cb28cd235671fe87eb4
I20260810 23:02:26.037611 5646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:26.041736 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.036018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.6.47.190:35175)
0810 23:02:26.036320 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.036328 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.036361 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:02:26.036484 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.036515 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.036591 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.036747 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:26.036996 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.038334 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.039428 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.040093 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.040223 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.040988 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:26.040993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:26.040993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:26.041206 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:26.041248 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:26.041262 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:26.041334 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.041407 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.041489 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":122,"threads_started":1}
I20260810 23:02:26.058645 5646 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:26.059345 5646 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:26.064519 5646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: Generated new TSK 0
I20260810 23:02:26.064720 5646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:26.066874 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--builtin_ntp_servers=127.6.47.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:26.150815 5673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:26.151067 5673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:26.151120 5673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:26.152331 5673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:26.152405 5673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:26.152447 5673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:26.153870 5673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:26.154183 5673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:26.154505 5673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:26.160508 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.160650 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.160528 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:26.160738 5673 server_base.cc:1034] running on GCE node
I20260810 23:02:26.160983 5673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:26.161234 5673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:26.162405 5673 hybrid_clock.cc:648] HybridClock initialized: now 1786402946162387 us; error 34 us; skew 500 ppm
I20260810 23:02:26.163703 5673 webserver.cc:460] Webserver started at http://127.6.47.129:41053/ using document root <none> and password file <none>
I20260810 23:02:26.163952 5673 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:26.164024 5673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:26.164194 5673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:26.165416 5673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "3ab61efdeb184feab6047f73f285d6e8"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "2e6a3952e1846d4fb7cdb980571d64c1"
server_key_iv: "804486c5b8505e0f82ef5f79244c0189"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.165817 5673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "3ab61efdeb184feab6047f73f285d6e8"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "2e6a3952e1846d4fb7cdb980571d64c1"
server_key_iv: "804486c5b8505e0f82ef5f79244c0189"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.167209 5673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260810 23:02:26.167975 5699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.168152 5673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:26.168265 5673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "3ab61efdeb184feab6047f73f285d6e8"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "2e6a3952e1846d4fb7cdb980571d64c1"
server_key_iv: "804486c5b8505e0f82ef5f79244c0189"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.168344 5673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:26.178920 5673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:26.179248 5673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:26.179409 5673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:26.179656 5673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:26.179998 5673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:26.180054 5673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.180099 5673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:26.180122 5673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.186266 5673 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:45719
I20260810 23:02:26.186342 5878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:45719 every 8 connection(s)
I20260810 23:02:26.186720 5673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:26.191983 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.187484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:59149 (local address 127.6.47.190:35175)
0810 23:02:26.187548 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.187550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.187779 (+ 229us) server_negotiation.cc:407] Connection header received
0810 23:02:26.188261 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.188262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.188275 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.188328 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:26.189014 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.189652 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.190664 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.190848 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.190902 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.191765 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:26.191766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:26.191766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:26.191770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:26.191781 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:26.191784 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:26.191821 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.191881 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.191904 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 23:02:26.192211 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.187465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35175 (local address 127.6.47.129:59149)
0810 23:02:26.187645 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.187684 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.188160 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.188380 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.188383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.188392 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:26.188912 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.188935 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.189736 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.189737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.190590 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.190592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.190922 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.190923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.191031 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.191613 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:26.191613 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:26.191643 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:26.191678 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:26.191822 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:26.191824 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:26.191826 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.191871 (+ 45us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.191945 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260810 23:02:26.192765 5880 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35175
I20260810 23:02:26.192859 5880 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:26.192998 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 5673
I20260810 23:02:26.193055 5880 heartbeater.cc:511] Master 127.6.47.190:35175 requested a full tablet report, sending...
I20260810 23:02:26.193091 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:26.193288 6334 external_mini_cluster.cc:1428] Setting key 04401378cbae47659de793aa7d374eeb
I20260810 23:02:26.193632 5564 ts_manager.cc:194] Registered new tserver with Master: 3ab61efdeb184feab6047f73f285d6e8 (127.6.47.129:45719)
I20260810 23:02:26.194334 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--builtin_ntp_servers=127.6.47.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:02:26.194393 5564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.129:59149
W20260810 23:02:26.277614 5884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:26.277805 5884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:26.277827 5884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:26.278879 5884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:26.278914 5884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:26.278939 5884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:26.280336 5884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:26.280591 5884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:26.280857 5884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:02:26.286070 5884 server_base.cc:1034] running on GCE node
W20260810 23:02:26.285996 5893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.285964 5897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.285964 5891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:26.286454 5884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:26.286664 5884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:26.287810 5884 hybrid_clock.cc:648] HybridClock initialized: now 1786402946287790 us; error 35 us; skew 500 ppm
I20260810 23:02:26.289228 5884 webserver.cc:460] Webserver started at http://127.6.47.130:46523/ using document root <none> and password file <none>
I20260810 23:02:26.289529 5884 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:26.289605 5884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:26.289769 5884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:26.290958 5884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "b89cf0cebdea468f93d65da27383c41e"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e8e509f005add8507181dc00b5126850"
server_key_iv: "ea6b72b1f23a9e1ced7276222dfdd42e"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.291334 5884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "b89cf0cebdea468f93d65da27383c41e"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e8e509f005add8507181dc00b5126850"
server_key_iv: "ea6b72b1f23a9e1ced7276222dfdd42e"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.292701 5884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260810 23:02:26.293449 5910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.293618 5884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:26.293743 5884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "b89cf0cebdea468f93d65da27383c41e"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e8e509f005add8507181dc00b5126850"
server_key_iv: "ea6b72b1f23a9e1ced7276222dfdd42e"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.293816 5884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:26.303196 5884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:26.303484 5884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:26.303629 5884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:26.303839 5884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:26.304147 5884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:26.304211 5884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.304251 5884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:26.304280 5884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.310009 5884 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:38345
I20260810 23:02:26.310045 6106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:38345 every 8 connection(s)
I20260810 23:02:26.310374 5884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:02:26.315436 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.311181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:54007 (local address 127.6.47.190:35175)
0810 23:02:26.311282 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.311284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.311454 (+ 170us) server_negotiation.cc:407] Connection header received
0810 23:02:26.311882 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.311884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.311894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.311944 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:26.312631 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.313089 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.314098 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.314287 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.314342 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.315204 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:26.315205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:26.315205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:26.315209 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:26.315222 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:26.315225 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:26.315268 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.315355 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.315378 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 23:02:26.315665 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.311136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35175 (local address 127.6.47.130:54007)
0810 23:02:26.311328 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.311359 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.311799 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.311994 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.311997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.312004 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:26.312470 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.312474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.313175 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.313176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.314022 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.314024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.314364 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.314365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.314497 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.315071 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:26.315073 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:26.315106 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:26.315128 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:26.315284 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:26.315290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:26.315292 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.315343 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.315417 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":112,"threads_started":1}
I20260810 23:02:26.316184 6107 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35175
I20260810 23:02:26.316274 6107 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:26.316466 6107 heartbeater.cc:511] Master 127.6.47.190:35175 requested a full tablet report, sending...
I20260810 23:02:26.316875 5564 ts_manager.cc:194] Registered new tserver with Master: b89cf0cebdea468f93d65da27383c41e (127.6.47.130:38345)
I20260810 23:02:26.317224 5564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.130:54007
I20260810 23:02:26.319891 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 5884
I20260810 23:02:26.319970 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:26.320123 6334 external_mini_cluster.cc:1428] Setting key c2cf23da2f87f27a5babf62a9f38427a
I20260810 23:02:26.321082 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--builtin_ntp_servers=127.6.47.148:40973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:26.406378 6112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:26.406651 6112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:26.406673 6112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:26.407755 6112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:26.407788 6112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:26.407812 6112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:26.409164 6112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:26.409373 6112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:26.409638 6112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:26.414546 6119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.414608 6122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:26.414534 6125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:26.414921 6112 server_base.cc:1034] running on GCE node
I20260810 23:02:26.415191 6112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:26.415390 6112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:26.416570 6112 hybrid_clock.cc:648] HybridClock initialized: now 1786402946416547 us; error 27 us; skew 500 ppm
I20260810 23:02:26.417996 6112 webserver.cc:460] Webserver started at http://127.6.47.131:41637/ using document root <none> and password file <none>
I20260810 23:02:26.418355 6112 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:26.418439 6112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:26.418630 6112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:26.419782 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "00c65eb461244866bf5347dd9d69439a"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e61c9b42c44400110c43fc795418fb47"
server_key_iv: "d648c95b4ff3868a7bb929e8d1fab209"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.420341 6112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "00c65eb461244866bf5347dd9d69439a"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e61c9b42c44400110c43fc795418fb47"
server_key_iv: "d648c95b4ff3868a7bb929e8d1fab209"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.421785 6112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:26.422576 6142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.422750 6112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:26.422852 6112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "00c65eb461244866bf5347dd9d69439a"
format_stamp: "Formatted at 2026-08-10 23:02:26 on dist-test-slave-rwrg"
server_key: "e61c9b42c44400110c43fc795418fb47"
server_key_iv: "d648c95b4ff3868a7bb929e8d1fab209"
server_key_version: "encryptionkey@0"
I20260810 23:02:26.422936 6112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:26.435658 6112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:26.435940 6112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:26.436091 6112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:26.436321 6112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:26.436712 6112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:26.436766 6112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.436806 6112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:26.436832 6112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:26.442621 6112 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:38827
I20260810 23:02:26.442685 6349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:38827 every 8 connection(s)
I20260810 23:02:26.443008 6112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:26.447155 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6112
I20260810 23:02:26.447275 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:26.447484 6334 external_mini_cluster.cc:1428] Setting key cc36b168ee6e2a3b2669d6537e32d16d
I20260810 23:02:26.448652 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.443747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:47707 (local address 127.6.47.190:35175)
0810 23:02:26.443823 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.443825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.444077 (+ 252us) server_negotiation.cc:407] Connection header received
0810 23:02:26.444693 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.444694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.444707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.444770 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:26.445451 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.445953 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.446917 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.447152 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.447239 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.448217 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:26.448218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:26.448218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:26.448223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:26.448240 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:26.448244 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:26.448322 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.448418 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.448462 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:02:26.448789 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.443753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35175 (local address 127.6.47.131:47707)
0810 23:02:26.443950 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.443984 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.444567 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.444855 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.444858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.444866 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:26.445346 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.445351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.446073 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.446074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.446825 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.446827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.447252 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.447254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.447427 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.448058 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:26.448059 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:26.448100 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:26.448131 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:26.448335 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:26.448342 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:26.448346 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.448416 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.448545 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260810 23:02:26.449255 6350 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35175
I20260810 23:02:26.449364 6350 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:26.449565 6350 heartbeater.cc:511] Master 127.6.47.190:35175 requested a full tablet report, sending...
I20260810 23:02:26.449934 5564 ts_manager.cc:194] Registered new tserver with Master: 00c65eb461244866bf5347dd9d69439a (127.6.47.131:38827)
I20260810 23:02:26.450309 5564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.131:47707
I20260810 23:02:26.459465 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:26.464073 5666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.460941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.6.47.190:35175)
0810 23:02:26.461029 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.461031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.461148 (+ 117us) server_negotiation.cc:407] Connection header received
0810 23:02:26.461277 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.461278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.461291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.461350 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:26.461524 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.462101 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.462867 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.463072 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.463148 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.463842 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:26.463843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:26.463844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:26.463849 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:26.463864 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:26.463868 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:26.463913 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.463968 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.463991 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:02:26.465863 5564 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:26.467407 5564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:26.473803 6365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.469710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45719 (local address 127.0.0.1:43374)
0810 23:02:26.470270 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.470282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.470326 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.470638 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.470639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.470640 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:26.470694 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.470696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471903 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.471904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.472759 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.472762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.473496 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.473497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.473572 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.473587 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.473668 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.473721 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260810 23:02:26.473819 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.469835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.6.47.129:45719)
0810 23:02:26.470269 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.470273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.470388 (+ 115us) server_negotiation.cc:407] Connection header received
0810 23:02:26.470476 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.470479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.470496 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.470567 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:26.470773 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471820 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.472841 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.473405 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.473481 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.473569 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.473697 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.473737 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":102,"threads_started":1}
I20260810 23:02:26.474263 6372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.469936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:38345 (local address 127.0.0.1:47676)
0810 23:02:26.470329 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.470340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.470397 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.470625 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.470626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.470628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:26.470675 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.470677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471623 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.471624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.472833 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.472836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.473789 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.473790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.473848 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.473851 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.474158 (+ 307us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.474202 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":38,"thread_start_us":80,"threads_started":1}
I20260810 23:02:26.474323 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.470046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.6.47.130:38345)
0810 23:02:26.470216 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.470220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.470473 (+ 253us) server_negotiation.cc:407] Connection header received
0810 23:02:26.470520 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.470523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.470539 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.470628 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:26.470729 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471506 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.472922 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.473618 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.473676 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.473756 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.474198 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.474244 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260810 23:02:26.474618 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.469710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:38827 (local address 127.0.0.1:45936)
0810 23:02:26.470036 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.470079 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.470154 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.470418 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.470421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.470423 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:26.470503 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.470504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471868 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.471869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.473194 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.473196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.474262 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.474263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.474468 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.474472 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.474507 (+ 35us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.474545 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260810 23:02:26.474606 5767 tablet_service.cc:1467] Processing CreateTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 (DEFAULT_TABLE table=test-table [id=b722ba0cf39d41f0965f5beff63142c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:26.474984 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.469851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.6.47.131:38827)
0810 23:02:26.470049 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.470054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.470132 (+ 78us) server_negotiation.cc:407] Connection header received
0810 23:02:26.470237 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.470240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.470256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.470583 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:26.470640 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.471781 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.473281 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.474158 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.474742 (+ 584us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.474832 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.474858 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.474896 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
I20260810 23:02:26.475006 5767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adc20deeb594820906ba0f47fb1ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:26.474942 6000 tablet_service.cc:1467] Processing CreateTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 (DEFAULT_TABLE table=test-table [id=b722ba0cf39d41f0965f5beff63142c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:26.475301 6000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adc20deeb594820906ba0f47fb1ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:26.475610 6218 tablet_service.cc:1467] Processing CreateTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 (DEFAULT_TABLE table=test-table [id=b722ba0cf39d41f0965f5beff63142c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:26.475940 6218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adc20deeb594820906ba0f47fb1ac01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:26.477953 6374 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Bootstrap starting.
I20260810 23:02:26.478873 6374 tablet_bootstrap.cc:654] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:26.479177 6374 log.cc:824] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:26.479261 6377 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Bootstrap starting.
I20260810 23:02:26.479532 6376 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Bootstrap starting.
I20260810 23:02:26.479874 6374 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: No bootstrap required, opened a new log
I20260810 23:02:26.479929 6377 tablet_bootstrap.cc:654] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:26.479952 6374 ts_tablet_manager.cc:1397] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:02:26.480209 6376 tablet_bootstrap.cc:654] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:26.480280 6377 log.cc:824] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:26.480578 6376 log.cc:824] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:26.481069 6377 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: No bootstrap required, opened a new log
I20260810 23:02:26.481149 6377 ts_tablet_manager.cc:1397] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:02:26.482353 6374 raft_consensus.cc:348] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.482489 6374 raft_consensus.cc:374] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:26.482511 6374 raft_consensus.cc:729] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b89cf0cebdea468f93d65da27383c41e, State: Initialized, Role: FOLLOWER
I20260810 23:02:26.482643 6374 consensus_queue.cc:260] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.482905 6374 ts_tablet_manager.cc:1428] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:02:26.482988 6107 heartbeater.cc:503] Master 127.6.47.190:35175 was elected leader, sending a full tablet report...
I20260810 23:02:26.483733 6377 raft_consensus.cc:348] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.483903 6376 tablet_bootstrap.cc:492] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: No bootstrap required, opened a new log
I20260810 23:02:26.483891 6377 raft_consensus.cc:374] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:26.483934 6377 raft_consensus.cc:729] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00c65eb461244866bf5347dd9d69439a, State: Initialized, Role: FOLLOWER
I20260810 23:02:26.483959 6376 ts_tablet_manager.cc:1397] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260810 23:02:26.484104 6377 consensus_queue.cc:260] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.484417 6377 ts_tablet_manager.cc:1428] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:02:26.485941 6376 raft_consensus.cc:348] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.486140 6376 raft_consensus.cc:374] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:26.486186 6376 raft_consensus.cc:729] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab61efdeb184feab6047f73f285d6e8, State: Initialized, Role: FOLLOWER
I20260810 23:02:26.486191 6350 heartbeater.cc:503] Master 127.6.47.190:35175 was elected leader, sending a full tablet report...
I20260810 23:02:26.486306 6376 consensus_queue.cc:260] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.486572 6376 ts_tablet_manager.cc:1428] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:02:26.486647 5880 heartbeater.cc:503] Master 127.6.47.190:35175 was elected leader, sending a full tablet report...
W20260810 23:02:26.561020 6108 tablet.cc:2369] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:02:26.687443 5882 tablet.cc:2369] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 23:02:26.693632 6352 tablet.cc:2369] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:02:26.753541 6384 raft_consensus.cc:482] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:26.753671 6384 raft_consensus.cc:504] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.754034 6384 leader_election.cc:288] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00c65eb461244866bf5347dd9d69439a (127.6.47.131:38827), 3ab61efdeb184feab6047f73f285d6e8 (127.6.47.129:45719)
I20260810 23:02:26.756906 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.754168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:38827 (local address 127.6.47.130:35895)
0810 23:02:26.754232 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.754246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.754295 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.754519 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.754519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.754520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:26.754578 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.754579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.755278 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.755279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.756033 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.756035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.756632 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.756715 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.756718 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.756783 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.756834 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 23:02:26.756906 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.754262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:35895 (local address 127.6.47.131:38827)
0810 23:02:26.754329 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.754331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.754341 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:26.754375 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.754376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.754385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.754480 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:26.754678 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.755184 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756134 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.756553 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756632 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.756641 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.756789 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.756834 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:02:26.757074 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.754400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:44133 (local address 127.6.47.129:45719)
0810 23:02:26.754483 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.754486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.754494 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:02:26.754520 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.754521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.754532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.754603 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:26.754809 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.755279 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756209 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.756674 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756765 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.756773 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.756942 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.756981 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 23:02:26.757206 6387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.754195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45719 (local address 127.6.47.130:44133)
0810 23:02:26.754384 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.754394 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:26.754447 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:26.754641 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:26.754643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:26.754645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:26.754732 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.754734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.755365 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.755366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.756102 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:26.756104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.756768 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.756769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:26.756866 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.756869 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:26.756937 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:02:26.757115 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
I20260810 23:02:26.757421 6261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adc20deeb594820906ba0f47fb1ac01" candidate_uuid: "b89cf0cebdea468f93d65da27383c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00c65eb461244866bf5347dd9d69439a" is_pre_election: true
I20260810 23:02:26.757597 6261 raft_consensus.cc:2454] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b89cf0cebdea468f93d65da27383c41e in term 0.
I20260810 23:02:26.757580 5801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adc20deeb594820906ba0f47fb1ac01" candidate_uuid: "b89cf0cebdea468f93d65da27383c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab61efdeb184feab6047f73f285d6e8" is_pre_election: true
I20260810 23:02:26.757764 5801 raft_consensus.cc:2454] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b89cf0cebdea468f93d65da27383c41e in term 0.
I20260810 23:02:26.757838 5912 leader_election.cc:302] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00c65eb461244866bf5347dd9d69439a, b89cf0cebdea468f93d65da27383c41e; no voters:
I20260810 23:02:26.757977 6384 raft_consensus.cc:2781] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:26.758057 6384 raft_consensus.cc:482] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:26.758076 6384 raft_consensus.cc:3037] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:26.758857 6384 raft_consensus.cc:504] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.759003 6384 leader_election.cc:288] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [CANDIDATE]: Term 1 election: Requested vote from peers 00c65eb461244866bf5347dd9d69439a (127.6.47.131:38827), 3ab61efdeb184feab6047f73f285d6e8 (127.6.47.129:45719)
I20260810 23:02:26.759166 5801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adc20deeb594820906ba0f47fb1ac01" candidate_uuid: "b89cf0cebdea468f93d65da27383c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab61efdeb184feab6047f73f285d6e8"
I20260810 23:02:26.759269 6261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adc20deeb594820906ba0f47fb1ac01" candidate_uuid: "b89cf0cebdea468f93d65da27383c41e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00c65eb461244866bf5347dd9d69439a"
I20260810 23:02:26.759276 5801 raft_consensus.cc:3037] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:26.759349 6261 raft_consensus.cc:3037] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:26.760298 6261 raft_consensus.cc:2454] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b89cf0cebdea468f93d65da27383c41e in term 1.
I20260810 23:02:26.760298 5801 raft_consensus.cc:2454] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b89cf0cebdea468f93d65da27383c41e in term 1.
I20260810 23:02:26.760471 5911 leader_election.cc:302] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ab61efdeb184feab6047f73f285d6e8, b89cf0cebdea468f93d65da27383c41e; no voters:
I20260810 23:02:26.760588 6384 raft_consensus.cc:2781] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:26.760727 6384 raft_consensus.cc:686] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 1 LEADER]: Becoming Leader. State: Replica: b89cf0cebdea468f93d65da27383c41e, State: Running, Role: LEADER
I20260810 23:02:26.760818 6384 consensus_queue.cc:237] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } }
I20260810 23:02:26.761726 5564 catalog_manager.cc:5495] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e reported cstate change: term changed from 0 to 1, leader changed from <none> to b89cf0cebdea468f93d65da27383c41e (127.6.47.130). New cstate: current_term: 1 leader_uuid: "b89cf0cebdea468f93d65da27383c41e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b89cf0cebdea468f93d65da27383c41e" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 38345 } health_report { overall_health: HEALTHY } } }
I20260810 23:02:26.826186 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.823815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.6.47.130:38345)
0810 23:02:26.823872 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:26.823875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:26.823885 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:26.823954 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:26.823955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:26.823973 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:26.824041 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:26.824178 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.824704 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.825487 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:26.825695 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:26.825747 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:26.825880 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:26.825985 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:26.826073 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:26.826096 (+ 23us) server_negotiation.cc:299] Negotiation successful
0810 23:02:26.826113 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:02:26.828081 6261 raft_consensus.cc:1261] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 1 FOLLOWER]: Refusing update from remote peer b89cf0cebdea468f93d65da27383c41e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:26.828081 5801 raft_consensus.cc:1261] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 1 FOLLOWER]: Refusing update from remote peer b89cf0cebdea468f93d65da27383c41e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:26.828539 6384 consensus_queue.cc:1035] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [LEADER]: Connected to new peer: Peer: permanent_uuid: "00c65eb461244866bf5347dd9d69439a" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 38827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:26.828747 6388 consensus_queue.cc:1035] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab61efdeb184feab6047f73f285d6e8" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:26.831420 6393 mvcc.cc:205] Tried to move back new op lower bound from 7317106470204841984 to 7317106469932908544. Current Snapshot: MvccSnapshot[applied={T|T < 7317106470204841984}]
I20260810 23:02:26.834057 5564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:26.834162 5564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:26.834995 5564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 683af1f7534d45dabd4b8ea1dc00c3ea: Sending DeleteTablet for 3 replicas of tablet 7adc20deeb594820906ba0f47fb1ac01
I20260810 23:02:26.835354 6218 tablet_service.cc:1514] Processing DeleteTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:26 UTC) from {username='slave'} at 127.0.0.1:45936
I20260810 23:02:26.835382 6000 tablet_service.cc:1514] Processing DeleteTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:26 UTC) from {username='slave'} at 127.0.0.1:47676
I20260810 23:02:26.835445 5767 tablet_service.cc:1514] Processing DeleteTablet for tablet 7adc20deeb594820906ba0f47fb1ac01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:26 UTC) from {username='slave'} at 127.0.0.1:43374
I20260810 23:02:26.835635 6410 tablet_replica.cc:330] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: stopping tablet replica
I20260810 23:02:26.835738 6412 tablet_replica.cc:330] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8: stopping tablet replica
I20260810 23:02:26.835803 6410 raft_consensus.cc:2229] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:26.835891 6412 raft_consensus.cc:2229] T 7adc20deeb594820906ba0f47fb1ac01 P 3ab61efdeb184feab6047f73f285d6e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:26.835980 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 5673
I20260810 23:02:26.835995 6410 raft_consensus.cc:2258] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:26.843642 6410 ts_tablet_manager.cc:1905] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:26.845446 6410 ts_tablet_manager.cc:1918] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:26.845448 6409 tablet_replica.cc:330] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: stopping tablet replica
I20260810 23:02:26.845582 6409 raft_consensus.cc:2229] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:26.845621 6410 log.cc:1196] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/7adc20deeb594820906ba0f47fb1ac01
I20260810 23:02:26.845816 6409 raft_consensus.cc:2258] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:26.845911 6410 ts_tablet_manager.cc:1939] T 7adc20deeb594820906ba0f47fb1ac01 P b89cf0cebdea468f93d65da27383c41e: Deleting consensus metadata
I20260810 23:02:26.846391 5538 catalog_manager.cc:4841] TS b89cf0cebdea468f93d65da27383c41e (127.6.47.130:38345): tablet 7adc20deeb594820906ba0f47fb1ac01 (table test-table [id=b722ba0cf39d41f0965f5beff63142c5]) successfully deleted
W20260810 23:02:26.852275 5537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:45719 (error 108)
I20260810 23:02:26.852770 6409 ts_tablet_manager.cc:1905] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:26.852871 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:26.852722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45719 (local address 127.0.0.1:43384)
0810 23:02:26.852785 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:26.852829 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 23:02:26.852988 5537 catalog_manager.cc:4584] TS 3ab61efdeb184feab6047f73f285d6e8 (127.6.47.129:45719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7adc20deeb594820906ba0f47fb1ac01: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45719: connect: Connection refused (error 111)
I20260810 23:02:26.853274 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 5884
I20260810 23:02:26.854741 6409 ts_tablet_manager.cc:1918] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:26.854846 6409 log.cc:1196] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/7adc20deeb594820906ba0f47fb1ac01
I20260810 23:02:26.855083 6409 ts_tablet_manager.cc:1939] T 7adc20deeb594820906ba0f47fb1ac01 P 00c65eb461244866bf5347dd9d69439a: Deleting consensus metadata
I20260810 23:02:26.855846 5538 catalog_manager.cc:4841] TS 00c65eb461244866bf5347dd9d69439a (127.6.47.131:38827): tablet 7adc20deeb594820906ba0f47fb1ac01 (table test-table [id=b722ba0cf39d41f0965f5beff63142c5]) successfully deleted
I20260810 23:02:26.871289 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6112
I20260810 23:02:26.883455 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 5508
2026-08-10T23:02:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1023 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:26 dist-test-slave-rwrg krb5kdc[6421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:26 dist-test-slave-rwrg krb5kdc[6421](info): set up 2 sockets
Aug 10 23:02:26 dist-test-slave-rwrg krb5kdc[6421](info): commencing operation
krb5kdc: starting...
W20260810 23:02:28.168440 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:28.287811 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41669
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:28.370869 6441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:28.371083 6441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:28.371130 6441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:28.372305 6441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:28.372349 6441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:28.372363 6441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:28.372376 6441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:28.372423 6441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:28.373934 6441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:28.374174 6441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:28.374410 6441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:28.379455 6451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:28.379427 6453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.379504 6441 server_base.cc:1034] running on GCE node
W20260810 23:02:28.379434 6456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.379988 6441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:28.380297 6441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:28.381462 6441 hybrid_clock.cc:648] HybridClock initialized: now 1786402948381446 us; error 33 us; skew 500 ppm
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:28.383944 6441 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:28.384387 6441 webserver.cc:460] Webserver started at http://127.6.47.190:39879/ using document root <none> and password file <none>
I20260810 23:02:28.384665 6441 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:28.384735 6441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:28.384910 6441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:28.385982 6441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "a356d1de7e5d4150b82315dbc95ee818"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "aa49fa6f5e78e727886a7625508a1213"
server_key_iv: "11d9b36a1c920acb2f947eda643881c0"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.386368 6441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "a356d1de7e5d4150b82315dbc95ee818"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "aa49fa6f5e78e727886a7625508a1213"
server_key_iv: "11d9b36a1c920acb2f947eda643881c0"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.387785 6441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:28.388523 6467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.388703 6441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:28.388809 6441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "a356d1de7e5d4150b82315dbc95ee818"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "aa49fa6f5e78e727886a7625508a1213"
server_key_iv: "11d9b36a1c920acb2f947eda643881c0"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.388893 6441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:28.402493 6441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:28.404789 6441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:28.405022 6441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:28.409574 6441 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:41669
I20260810 23:02:28.409622 6568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:41669 every 8 connection(s)
I20260810 23:02:28.410012 6441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:28.410933 6569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:28.412720 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6441
I20260810 23:02:28.412840 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:28.413053 6334 external_mini_cluster.cc:1428] Setting key 8063d0457452cd0da2405c0f7aa03839
I20260810 23:02:28.413575 6569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: Bootstrap starting.
I20260810 23:02:28.414206 6569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:28.414520 6569 log.cc:824] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:28.415252 6569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: No bootstrap required, opened a new log
I20260810 23:02:28.417148 6569 raft_consensus.cc:348] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } }
I20260810 23:02:28.417371 6569 raft_consensus.cc:374] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:28.417464 6569 raft_consensus.cc:729] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a356d1de7e5d4150b82315dbc95ee818, State: Initialized, Role: FOLLOWER
I20260810 23:02:28.417631 6569 consensus_queue.cc:260] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } }
I20260810 23:02:28.417742 6569 raft_consensus.cc:388] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:28.417786 6569 raft_consensus.cc:482] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:28.417901 6569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:28.418453 6569 raft_consensus.cc:504] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } }
I20260810 23:02:28.418576 6569 leader_election.cc:302] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a356d1de7e5d4150b82315dbc95ee818; no voters:
I20260810 23:02:28.418735 6569 leader_election.cc:288] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:28.418780 6576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:28.418936 6576 raft_consensus.cc:686] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [term 1 LEADER]: Becoming Leader. State: Replica: a356d1de7e5d4150b82315dbc95ee818, State: Running, Role: LEADER
I20260810 23:02:28.419003 6569 sys_catalog.cc:564] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:28.419075 6576 consensus_queue.cc:237] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } }
I20260810 23:02:28.419379 6576 sys_catalog.cc:455] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a356d1de7e5d4150b82315dbc95ee818. Latest consensus state: current_term: 1 leader_uuid: "a356d1de7e5d4150b82315dbc95ee818" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } } }
I20260810 23:02:28.419484 6576 sys_catalog.cc:458] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:28.419394 6584 sys_catalog.cc:455] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a356d1de7e5d4150b82315dbc95ee818" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a356d1de7e5d4150b82315dbc95ee818" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41669 } } }
I20260810 23:02:28.419812 6584 sys_catalog.cc:458] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818 [sys.catalog]: This master's current role is: LEADER
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
W20260810 23:02:28.420966 6609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:02:28.421029 6609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:02:28.421077 6591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:28.421229 6591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:28.422873 6591 catalog_manager.cc:1347] Generated new cluster ID: 519683be83794d3f9d92e0c79a993075
I20260810 23:02:28.422921 6591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:28.425000 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.413632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.6.47.190:41669)
0810 23:02:28.413810 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.413821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.413860 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:02:28.414398 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.414405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.414457 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.414630 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:28.415346 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.416322 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.417291 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.417847 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.417968 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.421085 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:28.421089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:28.421090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:28.421101 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:28.423302 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.423771 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.423774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.423775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.423872 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.424087 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.424088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.424088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.424321 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:28.424524 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.424635 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.424773 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260810 23:02:28.438927 6591 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:28.439456 6591 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:28.447576 6591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: Generated new TSK 0
I20260810 23:02:28.447777 6591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:02:28.475844 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:28.558048 6617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:28.558257 6617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:28.558281 6617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:28.559445 6617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:28.559490 6617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:28.559517 6617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:28.561120 6617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:28.561352 6617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:28.561576 6617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:28.566632 6624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:28.566638 6622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:28.566642 6629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.566944 6617 server_base.cc:1034] running on GCE node
I20260810 23:02:28.567174 6617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:28.567417 6617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:28.568573 6617 hybrid_clock.cc:648] HybridClock initialized: now 1786402948568555 us; error 34 us; skew 500 ppm
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:28.574299 6617 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:02:28.574757 6617 webserver.cc:460] Webserver started at http://127.6.47.129:41787/ using document root <none> and password file <none>
I20260810 23:02:28.575004 6617 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:28.575073 6617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:28.575227 6617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:28.576251 6617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "32bd7bab9beb4121a3bfb214adb5fb10"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "b17c4011802ae30a785025962700c921"
server_key_iv: "4e28b16cd3804d78db22fdeb8b72ad64"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.576673 6617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "32bd7bab9beb4121a3bfb214adb5fb10"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "b17c4011802ae30a785025962700c921"
server_key_iv: "4e28b16cd3804d78db22fdeb8b72ad64"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.578032 6617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:28.578789 6644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.578958 6617 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:28.579063 6617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "32bd7bab9beb4121a3bfb214adb5fb10"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "b17c4011802ae30a785025962700c921"
server_key_iv: "4e28b16cd3804d78db22fdeb8b72ad64"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.579150 6617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:28.597854 6617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:28.600324 6617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:28.600528 6617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:28.600791 6617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:28.601121 6617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:28.601179 6617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.601220 6617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:28.601249 6617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.607053 6617 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:33797
I20260810 23:02:28.607101 6848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:33797 every 8 connection(s)
I20260810 23:02:28.607484 6617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:28.611325 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6617
I20260810 23:02:28.611470 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:28.611696 6334 external_mini_cluster.cc:1428] Setting key 9b566a3baa00c920527a0fbc0d2ae30b
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:28.617386 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.608310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:56809 (local address 127.6.47.190:41669)
0810 23:02:28.608394 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.608396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.608575 (+ 179us) server_negotiation.cc:407] Connection header received
0810 23:02:28.609073 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.609075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.609084 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.609141 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:28.610057 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.610555 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.611558 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.611818 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.611925 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.614440 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:28.614442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:28.614442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:28.614450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:28.615899 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.616336 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.616337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.616338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.616401 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.616722 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.616723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.616723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.616839 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:28.616914 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.617206 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.617295 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:02:28.617573 6852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.608256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41669 (local address 127.6.47.129:56809)
0810 23:02:28.608422 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.608460 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.608986 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.609201 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.609204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.609785 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:28.609977 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.609981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.610638 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.610639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.611475 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.611477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.611940 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.611941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.612110 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.612727 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:28.612739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:28.614335 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:28.616019 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.616027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.616030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.616256 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.616475 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.616476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.616476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.616625 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.616915 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:28.616924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:28.617064 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.617168 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.617335 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260810 23:02:28.618114 6850 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41669
I20260810 23:02:28.618237 6850 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:28.618428 6850 heartbeater.cc:511] Master 127.6.47.190:41669 requested a full tablet report, sending...
I20260810 23:02:28.619040 6508 ts_manager.cc:194] Registered new tserver with Master: 32bd7bab9beb4121a3bfb214adb5fb10 (127.6.47.129:33797)
I20260810 23:02:28.619815 6508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:56809
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:02:28.639225 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:28.723416 6856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:28.723623 6856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:28.723645 6856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:28.724902 6856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:28.724946 6856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:28.724972 6856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:28.726402 6856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:28.726661 6856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:28.726914 6856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:28.732018 6872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:28.732026 6870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.732189 6856 server_base.cc:1034] running on GCE node
W20260810 23:02:28.732048 6874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.732546 6856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:28.732811 6856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:28.733999 6856 hybrid_clock.cc:648] HybridClock initialized: now 1786402948733986 us; error 30 us; skew 500 ppm
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:28.736693 6856 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:02:28.737079 6856 webserver.cc:460] Webserver started at http://127.6.47.130:40077/ using document root <none> and password file <none>
I20260810 23:02:28.737269 6856 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:28.737311 6856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:28.737401 6856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:28.738391 6856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "f6d5475a2fa941039660d08f4dd5c8e5"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0888d57a196600e6ec47f3f715dfd68d"
server_key_iv: "aa040ab1c8982396e88d77b8bbd05d3a"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.738713 6856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "f6d5475a2fa941039660d08f4dd5c8e5"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0888d57a196600e6ec47f3f715dfd68d"
server_key_iv: "aa040ab1c8982396e88d77b8bbd05d3a"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.740025 6856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:28.740796 6881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.740972 6856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:28.741057 6856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "f6d5475a2fa941039660d08f4dd5c8e5"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0888d57a196600e6ec47f3f715dfd68d"
server_key_iv: "aa040ab1c8982396e88d77b8bbd05d3a"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.741113 6856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:28.748479 6856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:28.750320 6856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:28.750432 6856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:28.750614 6856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:28.750867 6856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:28.750898 6856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.750919 6856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:28.750931 6856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.756619 6856 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:33863
I20260810 23:02:28.756702 7074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:33863 every 8 connection(s)
I20260810 23:02:28.757009 6856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:28.764727 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 6856
I20260810 23:02:28.764835 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:28.765059 6334 external_mini_cluster.cc:1428] Setting key 22a2ff50334c2accc66dd9dd3ff5fca7
I20260810 23:02:28.766652 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.757783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:36007 (local address 127.6.47.190:41669)
0810 23:02:28.757827 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.757829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.758059 (+ 230us) server_negotiation.cc:407] Connection header received
0810 23:02:28.758588 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.758590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.758602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.758646 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:28.759598 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.760101 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.761184 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.761384 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.761472 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.763645 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:28.763646 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:28.763647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:28.763653 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:28.765093 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.765598 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.765599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.765600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.765681 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.765994 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.765995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.765995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.766100 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:28.766157 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.766517 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.766580 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:02:28.766954 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.757754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41669 (local address 127.6.47.130:36007)
0810 23:02:28.757939 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.757975 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.758474 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.758737 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.758741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.759312 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:28.759496 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.759500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.760213 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.760214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.761078 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.761081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.761485 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.761487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.761613 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.762122 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:28.762131 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:28.763557 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:28.765216 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.765225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.765227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.765524 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.765771 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.765771 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.765772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.765920 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.766183 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:28.766191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:28.766382 (+ 191us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.766495 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.766676 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260810 23:02:28.767552 7077 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41669
I20260810 23:02:28.767637 7077 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:28.767869 7077 heartbeater.cc:511] Master 127.6.47.190:41669 requested a full tablet report, sending...
I20260810 23:02:28.768325 6508 ts_manager.cc:194] Registered new tserver with Master: f6d5475a2fa941039660d08f4dd5c8e5 (127.6.47.130:33863)
I20260810 23:02:28.768851 6508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:36007
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:02:28.791950 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:43685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:28.877755 7091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:28.877965 7091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:28.877990 7091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:28.879148 7091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:28.879189 7091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:28.879215 7091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:28.880645 7091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:41669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:28.880831 7091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:28.881002 7091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:28.885939 7106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:28.885952 7109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.885969 7091 server_base.cc:1034] running on GCE node
W20260810 23:02:28.885967 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:28.886389 7091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:28.886588 7091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:28.887727 7091 hybrid_clock.cc:648] HybridClock initialized: now 1786402948887716 us; error 45 us; skew 500 ppm
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:28.890224 7091 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:02:28.890686 7091 webserver.cc:460] Webserver started at http://127.6.47.131:39233/ using document root <none> and password file <none>
I20260810 23:02:28.890941 7091 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:28.891011 7091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:28.891170 7091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:28.892297 7091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "9be9c85db5644803a9949c8200f58740"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0f1483453bf95dd0a4d1101fb56bde55"
server_key_iv: "6add674935b4cac622deb74e9ff83e6f"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.892706 7091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "9be9c85db5644803a9949c8200f58740"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0f1483453bf95dd0a4d1101fb56bde55"
server_key_iv: "6add674935b4cac622deb74e9ff83e6f"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.894006 7091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:28.894695 7124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.894889 7091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:28.894976 7091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "9be9c85db5644803a9949c8200f58740"
format_stamp: "Formatted at 2026-08-10 23:02:28 on dist-test-slave-rwrg"
server_key: "0f1483453bf95dd0a4d1101fb56bde55"
server_key_iv: "6add674935b4cac622deb74e9ff83e6f"
server_key_version: "encryptionkey@0"
I20260810 23:02:28.895030 7091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:28.903363 7091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:28.905720 7091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:28.905834 7091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:28.906035 7091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:28.906301 7091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:28.906333 7091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.906354 7091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:28.906368 7091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:28.912211 7091 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:33211
I20260810 23:02:28.912276 7321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:33211 every 8 connection(s)
I20260810 23:02:28.912634 7091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:28.917448 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7091
I20260810 23:02:28.917559 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:28.917769 6334 external_mini_cluster.cc:1428] Setting key 253ea96f11d377fa8efb3a359f41f47f
Aug 10 23:02:28 dist-test-slave-rwrg krb5kdc[6421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402948, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:28.922266 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.913389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:33911 (local address 127.6.47.190:41669)
0810 23:02:28.913450 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.913452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.913665 (+ 213us) server_negotiation.cc:407] Connection header received
0810 23:02:28.914132 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.914133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.914143 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.914194 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:28.915155 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.915669 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.916705 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.916880 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.916928 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.919512 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:28.919514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:28.919514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:28.919521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:28.921078 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.921466 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.921467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.921467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.921531 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.921747 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.921747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.921748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.921873 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:28.921922 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.922136 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.922201 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:02:28.922430 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.913346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41669 (local address 127.6.47.131:33911)
0810 23:02:28.913533 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.913576 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.914046 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.914277 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.914280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.914876 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:28.915069 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.915074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.915753 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.915754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.916626 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.916629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.916947 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.916948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.917050 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.917641 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:28.917652 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:28.919420 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:28.921163 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.921176 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.921177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.921396 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.921590 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:28.921590 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:28.921591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:28.921691 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:28.921914 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:28.921916 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:28.922024 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.922100 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.922229 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260810 23:02:28.922942 7322 heartbeater.cc:347] Connected to a master server at 127.6.47.190:41669
I20260810 23:02:28.923056 7322 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:28.923255 7322 heartbeater.cc:511] Master 127.6.47.190:41669 requested a full tablet report, sending...
I20260810 23:02:28.923702 6508 ts_manager.cc:194] Registered new tserver with Master: 9be9c85db5644803a9949c8200f58740 (127.6.47.131:33211)
I20260810 23:02:28.924104 6508 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:33911
I20260810 23:02:28.929586 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:28.936743 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.930899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.6.47.190:41669)
0810 23:02:28.930959 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.930962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.931087 (+ 125us) server_negotiation.cc:407] Connection header received
0810 23:02:28.931131 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.931132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.931140 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.931182 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:28.931667 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.932119 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.932908 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.933095 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.933161 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.934373 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:28.934375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:28.934375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:28.934381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:28.935561 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.935934 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.935935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.935935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.935985 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:28.936171 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:28.936172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:28.936172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:28.936293 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:28.936372 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.936539 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.936618 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:02:28.938289 6508 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:28.939860 6508 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:28.946502 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.6.47.131:33211)
0810 23:02:28.942692 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.942713 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.942724 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:02:28.942765 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.942768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.942785 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.943049 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:28.943130 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.944202 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.945438 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.946186 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946268 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946322 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.946415 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.946445 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":83,"threads_started":1}
I20260810 23:02:28.946522 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:33211 (local address 127.0.0.1:40974)
0810 23:02:28.942502 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.942514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.942569 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.942890 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.942891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.942893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:28.942966 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.942968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.944345 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.944346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.945349 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.945352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.946277 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.946333 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946336 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.946396 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.946426 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":21,"thread_start_us":70,"threads_started":1}
I20260810 23:02:28.946691 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:33863 (local address 127.0.0.1:37302)
0810 23:02:28.942655 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.942664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.942729 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.942918 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.942919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.942921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:28.942961 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.942961 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.944241 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.944242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.945442 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.945444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.946467 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.946533 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946537 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.946588 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.946626 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":57,"threads_started":1}
I20260810 23:02:28.946946 7344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.6.47.130:33863)
0810 23:02:28.942595 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.942600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.942784 (+ 184us) server_negotiation.cc:407] Connection header received
0810 23:02:28.942829 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.942831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.942845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.942920 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:28.943006 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.944134 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.945514 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.946195 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946268 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946322 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.946801 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.946846 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
I20260810 23:02:28.947118 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.6.47.129:33797)
0810 23:02:28.942487 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:28.942493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:28.942504 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:02:28.942578 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:28.942580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:28.942594 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:28.942720 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:28.943105 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.943903 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.945469 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.945993 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946096 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946161 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:28.946967 (+ 806us) server_negotiation.cc:299] Negotiation successful
0810 23:02:28.947016 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":116,"threads_started":1}
I20260810 23:02:28.946434 7341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:28.942061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:33797 (local address 127.0.0.1:52550)
0810 23:02:28.942372 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:28.942414 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:28.942482 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:28.942728 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:28.942731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:28.942735 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:28.942810 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.942813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.944005 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.944006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.945367 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:28.945371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:28.946100 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:28.946101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:28.946183 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:28.946199 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 23:02:28.946293 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 23:02:28.946345 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":58,"threads_started":1}
I20260810 23:02:28.947353 7202 tablet_service.cc:1467] Processing CreateTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 (DEFAULT_TABLE table=test-table [id=506673cb2f4246b6b711e3c295e58002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:28.947747 7202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440d2940db4f4218ae334f3ff406a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:28.947919 6740 tablet_service.cc:1467] Processing CreateTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 (DEFAULT_TABLE table=test-table [id=506673cb2f4246b6b711e3c295e58002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:28.948276 6740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440d2940db4f4218ae334f3ff406a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:28.949174 6970 tablet_service.cc:1467] Processing CreateTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 (DEFAULT_TABLE table=test-table [id=506673cb2f4246b6b711e3c295e58002]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:28.949545 6970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440d2940db4f4218ae334f3ff406a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:28.950989 7345 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Bootstrap starting.
I20260810 23:02:28.951647 7348 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: Bootstrap starting.
I20260810 23:02:28.951723 7345 tablet_bootstrap.cc:654] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:28.952364 7345 log.cc:824] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:28.952395 7348 tablet_bootstrap.cc:654] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:28.952700 7352 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Bootstrap starting.
I20260810 23:02:28.953063 7348 log.cc:824] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:28.953464 7352 tablet_bootstrap.cc:654] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:28.953563 7345 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: No bootstrap required, opened a new log
I20260810 23:02:28.953822 7345 ts_tablet_manager.cc:1397] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 23:02:28.953881 7348 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: No bootstrap required, opened a new log
I20260810 23:02:28.953948 7348 ts_tablet_manager.cc:1397] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:02:28.954087 7352 log.cc:824] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:28.954931 7352 tablet_bootstrap.cc:492] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: No bootstrap required, opened a new log
I20260810 23:02:28.955003 7352 ts_tablet_manager.cc:1397] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:28.955564 7345 raft_consensus.cc:348] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.955739 7345 raft_consensus.cc:374] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:28.955785 7345 raft_consensus.cc:729] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be9c85db5644803a9949c8200f58740, State: Initialized, Role: FOLLOWER
I20260810 23:02:28.955915 7345 consensus_queue.cc:260] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.956107 7348 raft_consensus.cc:348] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.956259 7345 ts_tablet_manager.cc:1428] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:28.956291 7348 raft_consensus.cc:374] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:28.956331 7348 raft_consensus.cc:729] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32bd7bab9beb4121a3bfb214adb5fb10, State: Initialized, Role: FOLLOWER
I20260810 23:02:28.956387 7322 heartbeater.cc:503] Master 127.6.47.190:41669 was elected leader, sending a full tablet report...
I20260810 23:02:28.956467 7348 consensus_queue.cc:260] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.956753 7348 ts_tablet_manager.cc:1428] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 23:02:28.956903 6850 heartbeater.cc:503] Master 127.6.47.190:41669 was elected leader, sending a full tablet report...
I20260810 23:02:28.957072 7352 raft_consensus.cc:348] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.957260 7352 raft_consensus.cc:374] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:28.957307 7352 raft_consensus.cc:729] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d5475a2fa941039660d08f4dd5c8e5, State: Initialized, Role: FOLLOWER
I20260810 23:02:28.957451 7352 consensus_queue.cc:260] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:28.957738 7352 ts_tablet_manager.cc:1428] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:02:28.957973 7077 heartbeater.cc:503] Master 127.6.47.190:41669 was elected leader, sending a full tablet report...
W20260810 23:02:29.007706 7079 tablet.cc:2369] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:02:29.038839 7365 raft_consensus.cc:482] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:29.038975 7365 raft_consensus.cc:504] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:29.039467 7365 leader_election.cc:288] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32bd7bab9beb4121a3bfb214adb5fb10 (127.6.47.129:33797), 9be9c85db5644803a9949c8200f58740 (127.6.47.131:33211)
I20260810 23:02:29.042349 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.039753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:42315 (local address 127.6.47.131:33211)
0810 23:02:29.039801 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.039804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.039834 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:02:29.039867 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.039868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.039880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.039960 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.040141 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.040640 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.041583 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.041983 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.042071 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.042081 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.042241 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.042281 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:02:29.042372 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.039651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:33211 (local address 127.6.47.130:42315)
0810 23:02:29.039715 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.039729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.039780 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.039974 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.039976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.039978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.040059 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.040061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.040737 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.040738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.041486 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.041488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.042071 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.042072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.042156 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.042159 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.042234 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.042279 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:02:29.042600 7368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.039667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:33797 (local address 127.6.47.130:50273)
0810 23:02:29.039926 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.039934 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.040004 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.040173 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.040174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.040176 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.040251 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.040253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.040968 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.040969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.041699 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.041700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.042287 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.042288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.042374 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.042377 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.042445 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.042493 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":66,"thread_start_us":133,"threads_started":1}
I20260810 23:02:29.042722 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.039782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:50273 (local address 127.6.47.129:33797)
0810 23:02:29.039826 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.039828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.040008 (+ 180us) server_negotiation.cc:407] Connection header received
0810 23:02:29.040064 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.040065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.040079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.040155 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.040324 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.040879 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.041805 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.042196 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.042285 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.042293 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.042581 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.042639 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:02:29.042752 7234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440d2940db4f4218ae334f3ff406a5c8" candidate_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be9c85db5644803a9949c8200f58740" is_pre_election: true
I20260810 23:02:29.042938 7234 raft_consensus.cc:2454] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6d5475a2fa941039660d08f4dd5c8e5 in term 0.
I20260810 23:02:29.043018 6771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440d2940db4f4218ae334f3ff406a5c8" candidate_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" is_pre_election: true
I20260810 23:02:29.043197 6771 raft_consensus.cc:2454] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6d5475a2fa941039660d08f4dd5c8e5 in term 0.
I20260810 23:02:29.043222 6885 leader_election.cc:302] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9be9c85db5644803a9949c8200f58740, f6d5475a2fa941039660d08f4dd5c8e5; no voters:
I20260810 23:02:29.043346 7365 raft_consensus.cc:2781] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:29.043432 7365 raft_consensus.cc:482] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:29.043464 7365 raft_consensus.cc:3037] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.044210 7365 raft_consensus.cc:504] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:29.044368 7365 leader_election.cc:288] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [CANDIDATE]: Term 1 election: Requested vote from peers 32bd7bab9beb4121a3bfb214adb5fb10 (127.6.47.129:33797), 9be9c85db5644803a9949c8200f58740 (127.6.47.131:33211)
I20260810 23:02:29.044570 6771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440d2940db4f4218ae334f3ff406a5c8" candidate_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32bd7bab9beb4121a3bfb214adb5fb10"
I20260810 23:02:29.044625 7234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440d2940db4f4218ae334f3ff406a5c8" candidate_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9be9c85db5644803a9949c8200f58740"
I20260810 23:02:29.044688 6771 raft_consensus.cc:3037] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.044699 7234 raft_consensus.cc:3037] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.045650 6771 raft_consensus.cc:2454] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6d5475a2fa941039660d08f4dd5c8e5 in term 1.
I20260810 23:02:29.045650 7234 raft_consensus.cc:2454] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6d5475a2fa941039660d08f4dd5c8e5 in term 1.
I20260810 23:02:29.045857 6886 leader_election.cc:302] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32bd7bab9beb4121a3bfb214adb5fb10, f6d5475a2fa941039660d08f4dd5c8e5; no voters:
I20260810 23:02:29.046000 7365 raft_consensus.cc:2781] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:29.046204 7365 raft_consensus.cc:686] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 1 LEADER]: Becoming Leader. State: Replica: f6d5475a2fa941039660d08f4dd5c8e5, State: Running, Role: LEADER
I20260810 23:02:29.046334 7365 consensus_queue.cc:237] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } }
I20260810 23:02:29.047189 6508 catalog_manager.cc:5495] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6d5475a2fa941039660d08f4dd5c8e5 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6d5475a2fa941039660d08f4dd5c8e5" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 33863 } health_report { overall_health: HEALTHY } } }
I20260810 23:02:29.062199 7344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.059679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.6.47.130:33863)
0810 23:02:29.059752 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.059754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.059764 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:29.059818 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.059818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.059837 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.059912 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:29.060071 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.060605 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.061381 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.061578 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.061656 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.061834 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:29.061925 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:29.062041 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.062071 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.062083 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 23:02:29.064123 7234 raft_consensus.cc:1261] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 1 FOLLOWER]: Refusing update from remote peer f6d5475a2fa941039660d08f4dd5c8e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:29.064162 6771 raft_consensus.cc:1261] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 1 FOLLOWER]: Refusing update from remote peer f6d5475a2fa941039660d08f4dd5c8e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:29.064559 7365 consensus_queue.cc:1035] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32bd7bab9beb4121a3bfb214adb5fb10" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 33797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:29.064755 7369 consensus_queue.cc:1035] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9be9c85db5644803a9949c8200f58740" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 33211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:29.067385 7373 mvcc.cc:205] Tried to move back new op lower bound from 7317106479363686400 to 7317106479294341120. Current Snapshot: MvccSnapshot[applied={T|T < 7317106479363686400}]
I20260810 23:02:29.067777 7376 mvcc.cc:205] Tried to move back new op lower bound from 7317106479363686400 to 7317106479294341120. Current Snapshot: MvccSnapshot[applied={T|T < 7317106479363686400}]
I20260810 23:02:29.069981 6507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:29.070089 6507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:29.070904 6507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a356d1de7e5d4150b82315dbc95ee818: Sending DeleteTablet for 3 replicas of tablet 440d2940db4f4218ae334f3ff406a5c8
I20260810 23:02:29.071278 6970 tablet_service.cc:1514] Processing DeleteTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:37302
I20260810 23:02:29.071305 6740 tablet_service.cc:1514] Processing DeleteTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:52550
I20260810 23:02:29.071408 7202 tablet_service.cc:1514] Processing DeleteTablet for tablet 440d2940db4f4218ae334f3ff406a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:40974
I20260810 23:02:29.071596 7394 tablet_replica.cc:330] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10: stopping tablet replica
I20260810 23:02:29.071709 7394 raft_consensus.cc:2229] T 440d2940db4f4218ae334f3ff406a5c8 P 32bd7bab9beb4121a3bfb214adb5fb10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:29.071772 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6617
I20260810 23:02:29.076774 7390 tablet_replica.cc:330] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: stopping tablet replica
I20260810 23:02:29.076922 7390 raft_consensus.cc:2229] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:29.077086 7390 raft_consensus.cc:2258] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:29.081861 7390 ts_tablet_manager.cc:1905] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:29.084615 7390 ts_tablet_manager.cc:1918] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:29.084733 7390 log.cc:1196] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/440d2940db4f4218ae334f3ff406a5c8
I20260810 23:02:29.085111 7390 ts_tablet_manager.cc:1939] T 440d2940db4f4218ae334f3ff406a5c8 P f6d5475a2fa941039660d08f4dd5c8e5: Deleting consensus metadata
I20260810 23:02:29.085657 6470 catalog_manager.cc:4841] TS f6d5475a2fa941039660d08f4dd5c8e5 (127.6.47.130:33863): tablet 440d2940db4f4218ae334f3ff406a5c8 (table test-table [id=506673cb2f4246b6b711e3c295e58002]) successfully deleted
I20260810 23:02:29.088681 7399 tablet_replica.cc:330] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: stopping tablet replica
I20260810 23:02:29.088907 7399 raft_consensus.cc:2229] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:29.089061 7399 raft_consensus.cc:2258] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:29.089663 7399 ts_tablet_manager.cc:1905] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:29.091730 7399 ts_tablet_manager.cc:1918] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:29.091826 7399 log.cc:1196] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/440d2940db4f4218ae334f3ff406a5c8
I20260810 23:02:29.092127 7399 ts_tablet_manager.cc:1939] T 440d2940db4f4218ae334f3ff406a5c8 P 9be9c85db5644803a9949c8200f58740: Deleting consensus metadata
I20260810 23:02:29.092681 6470 catalog_manager.cc:4841] TS 9be9c85db5644803a9949c8200f58740 (127.6.47.131:33211): tablet 440d2940db4f4218ae334f3ff406a5c8 (table test-table [id=506673cb2f4246b6b711e3c295e58002]) successfully deleted
W20260810 23:02:29.094019 6471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:33797 (error 108)
I20260810 23:02:29.094558 7341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.094402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:33797 (local address 127.0.0.1:52566)
0810 23:02:29.094472 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.094518 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:33797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:02:29.094661 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6856
W20260810 23:02:29.104065 6471 catalog_manager.cc:4584] TS 32bd7bab9beb4121a3bfb214adb5fb10 (127.6.47.129:33797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 440d2940db4f4218ae334f3ff406a5c8: Network error: Client connection negotiation failed: client connection to 127.6.47.129:33797: connect: Connection refused (error 111)
I20260810 23:02:29.105265 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7091
I20260810 23:02:29.118350 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 6441
2026-08-10T23:02:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2240 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T23:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:29Z Disabled control of system clock
I20260810 23:02:29.153955 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35105
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:36847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:35105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:29.237022 7408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:29.237226 7408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:29.237264 7408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:29.238472 7408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:29.238514 7408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:29.238534 7408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:29.238548 7408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:29.238595 7408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:29.239921 7408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:35105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:35105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:29.240165 7408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:29.240447 7408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:29.245348 7422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.245425 7416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.245368 7417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:29.245638 7408 server_base.cc:1034] running on GCE node
I20260810 23:02:29.245910 7408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:29.246241 7408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:29.247416 7408 hybrid_clock.cc:648] HybridClock initialized: now 1786402949247394 us; error 36 us; skew 500 ppm
I20260810 23:02:29.248698 7408 webserver.cc:460] Webserver started at http://127.6.47.190:34913/ using document root <none> and password file <none>
I20260810 23:02:29.248957 7408 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:29.249004 7408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:29.249183 7408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:29.250231 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "6e23f651b756474eadc10c4ea4304e1c"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "edd70b04d37b16988ebecdfbdf746131"
server_key_iv: "7cb14d2eb686da8aae77ed4e1b8bb497"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.250599 7408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "6e23f651b756474eadc10c4ea4304e1c"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "edd70b04d37b16988ebecdfbdf746131"
server_key_iv: "7cb14d2eb686da8aae77ed4e1b8bb497"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.251932 7408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:29.252717 7432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.252899 7408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:29.253002 7408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "6e23f651b756474eadc10c4ea4304e1c"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "edd70b04d37b16988ebecdfbdf746131"
server_key_iv: "7cb14d2eb686da8aae77ed4e1b8bb497"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.253089 7408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:29.274712 7408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:29.275064 7408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:29.275221 7408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:29.279469 7408 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:35105
I20260810 23:02:29.279506 7526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:35105 every 8 connection(s)
I20260810 23:02:29.279874 7408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:29.280931 7529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:29.283191 7529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: Bootstrap starting.
I20260810 23:02:29.283771 7529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:29.284027 7529 log.cc:824] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:29.284719 7529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: No bootstrap required, opened a new log
I20260810 23:02:29.286226 7529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } }
I20260810 23:02:29.286362 7529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:29.286384 7529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e23f651b756474eadc10c4ea4304e1c, State: Initialized, Role: FOLLOWER
I20260810 23:02:29.286525 7529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } }
I20260810 23:02:29.286607 7529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:29.286631 7529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:29.286659 7529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.287127 7529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } }
I20260810 23:02:29.287210 7529 leader_election.cc:302] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e23f651b756474eadc10c4ea4304e1c; no voters:
I20260810 23:02:29.287356 7529 leader_election.cc:288] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:29.287463 7536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:29.287650 7536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [term 1 LEADER]: Becoming Leader. State: Replica: 6e23f651b756474eadc10c4ea4304e1c, State: Running, Role: LEADER
I20260810 23:02:29.287684 7529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:29.287792 7536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } }
I20260810 23:02:29.288283 7538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e23f651b756474eadc10c4ea4304e1c. Latest consensus state: current_term: 1 leader_uuid: "6e23f651b756474eadc10c4ea4304e1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } } }
I20260810 23:02:29.288312 7537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e23f651b756474eadc10c4ea4304e1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e23f651b756474eadc10c4ea4304e1c" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 35105 } } }
I20260810 23:02:29.288532 7538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:29.288866 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7408
I20260810 23:02:29.288892 7548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:29.289040 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:29.289292 6334 external_mini_cluster.cc:1428] Setting key c7fd212ef9513cb2a494e7d1f55e4b1b
I20260810 23:02:29.288560 7537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:29.289587 7548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:29.291412 7548 catalog_manager.cc:1347] Generated new cluster ID: de1ea36519814cddae2d03a6a63dbfc3
I20260810 23:02:29.291491 7548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:29.296104 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.289843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.6.47.190:35105)
0810 23:02:29.290317 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.290327 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.290364 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:02:29.290463 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.290469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.290529 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.291180 (+ 651us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:29.291299 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.292988 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.293896 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.294364 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.294514 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.295340 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:29.295344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:29.295344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:29.295550 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:29.295588 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:29.295600 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:29.295681 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.295785 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.295873 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":80,"threads_started":1}
I20260810 23:02:29.296670 7548 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:29.297358 7548 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:29.302599 7548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: Generated new TSK 0
I20260810 23:02:29.302784 7548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:29.310027 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--builtin_ntp_servers=127.6.47.148:36847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:29.413937 7581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:29.414139 7581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:29.414160 7581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:29.415261 7581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:29.415300 7581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:29.415324 7581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:29.416791 7581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:29.417025 7581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:29.417344 7581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:29.422158 7609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.422340 7597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.422173 7607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:29.422515 7581 server_base.cc:1034] running on GCE node
I20260810 23:02:29.422780 7581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:29.423054 7581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:29.424214 7581 hybrid_clock.cc:648] HybridClock initialized: now 1786402949424202 us; error 27 us; skew 500 ppm
I20260810 23:02:29.425662 7581 webserver.cc:460] Webserver started at http://127.6.47.129:33321/ using document root <none> and password file <none>
I20260810 23:02:29.425908 7581 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:29.425954 7581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:29.426085 7581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:29.427242 7581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "b5180b4197e540d7a10f836ba60c3752"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "5b6461d4768ffe99a42a24d1e010512b"
server_key_iv: "6aa8bbe6f0f1399f305b9174b438909a"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.427615 7581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "b5180b4197e540d7a10f836ba60c3752"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "5b6461d4768ffe99a42a24d1e010512b"
server_key_iv: "6aa8bbe6f0f1399f305b9174b438909a"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.429069 7581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:29.429883 7624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.430038 7581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:29.430131 7581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "b5180b4197e540d7a10f836ba60c3752"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "5b6461d4768ffe99a42a24d1e010512b"
server_key_iv: "6aa8bbe6f0f1399f305b9174b438909a"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.430203 7581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:29.441884 7581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:29.442188 7581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:29.442332 7581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:29.442552 7581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:29.442911 7581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:29.442965 7581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.443007 7581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:29.443034 7581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.449255 7581 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:43107
I20260810 23:02:29.449337 7837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:43107 every 8 connection(s)
I20260810 23:02:29.449661 7581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:29.454533 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.450408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:41893 (local address 127.6.47.190:35105)
0810 23:02:29.450486 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.450488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.450754 (+ 266us) server_negotiation.cc:407] Connection header received
0810 23:02:29.451241 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.451242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.451252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.451309 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:29.451918 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.452377 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.453321 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.453499 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.453551 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.454298 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:29.454299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:29.454299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:29.454306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:29.454319 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:29.454322 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:29.454364 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.454452 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.454477 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 23:02:29.454726 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.450398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35105 (local address 127.6.47.129:41893)
0810 23:02:29.450609 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.450655 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.451160 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.451361 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.451364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.451372 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:29.451848 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.451852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.452469 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.452470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.453245 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.453247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.453574 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.453575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.453690 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.454147 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:29.454147 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:29.454175 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:29.454196 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:29.454379 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:29.454383 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:29.454386 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.454437 (+ 51us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.454514 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260810 23:02:29.455251 7838 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35105
I20260810 23:02:29.455359 7838 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:29.455545 7838 heartbeater.cc:511] Master 127.6.47.190:35105 requested a full tablet report, sending...
I20260810 23:02:29.456086 7463 ts_manager.cc:194] Registered new tserver with Master: b5180b4197e540d7a10f836ba60c3752 (127.6.47.129:43107)
I20260810 23:02:29.456776 7463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.129:41893
I20260810 23:02:29.456879 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7581
I20260810 23:02:29.456992 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:29.457224 6334 external_mini_cluster.cc:1428] Setting key 714e4bfe5ca5d4b38e000efbca3a7b01
I20260810 23:02:29.458117 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--builtin_ntp_servers=127.6.47.148:36847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:29.540400 7844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:29.540618 7844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:29.540640 7844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:29.541652 7844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:29.541687 7844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:29.541708 7844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:29.542943 7844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:29.543210 7844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:29.543510 7844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:29.548437 7853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.548439 7851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.548493 7849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:29.548744 7844 server_base.cc:1034] running on GCE node
I20260810 23:02:29.548951 7844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:29.549197 7844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:29.550379 7844 hybrid_clock.cc:648] HybridClock initialized: now 1786402949550382 us; error 52 us; skew 500 ppm
I20260810 23:02:29.551649 7844 webserver.cc:460] Webserver started at http://127.6.47.130:42791/ using document root <none> and password file <none>
I20260810 23:02:29.551946 7844 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:29.552018 7844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:29.552174 7844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:29.553218 7844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "334180d123ca1e71d484cf4dc72a9656"
server_key_iv: "ef3bbce6d013c86b80e3b10a0bbbd1db"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.553624 7844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "334180d123ca1e71d484cf4dc72a9656"
server_key_iv: "ef3bbce6d013c86b80e3b10a0bbbd1db"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.554963 7844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:29.555725 7868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.555904 7844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:29.556033 7844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "334180d123ca1e71d484cf4dc72a9656"
server_key_iv: "ef3bbce6d013c86b80e3b10a0bbbd1db"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.556105 7844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:29.565065 7844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:29.565299 7844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:29.565413 7844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:29.565598 7844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:29.565855 7844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:29.565884 7844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.565905 7844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:29.565917 7844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.571590 7844 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:41045
I20260810 23:02:29.571668 8086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:41045 every 8 connection(s)
I20260810 23:02:29.572002 7844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:02:29.574175 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 7844
I20260810 23:02:29.574292 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:29.574508 6334 external_mini_cluster.cc:1428] Setting key 196baafb09e0345bfeaee567ed00bc7c
I20260810 23:02:29.575474 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--builtin_ntp_servers=127.6.47.148:36847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 23:02:29.577692 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.572732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:45243 (local address 127.6.47.190:35105)
0810 23:02:29.572774 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.572776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.573030 (+ 254us) server_negotiation.cc:407] Connection header received
0810 23:02:29.573524 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.573525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.573535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.573596 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:29.574435 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.575066 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.576129 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.576436 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.576592 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.577451 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:29.577452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:29.577453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:29.577457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:29.577470 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:29.577474 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:29.577520 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.577599 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.577626 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:02:29.577955 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.572704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35105 (local address 127.6.47.130:45243)
0810 23:02:29.572880 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.572916 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.573425 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.573662 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.573665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.573673 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:29.574334 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.574341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.575172 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.575174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.576028 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.576031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.576578 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.576579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.576688 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.577257 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:29.577258 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:29.577319 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:29.577352 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:29.577523 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:29.577527 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:29.577530 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.577596 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.577706 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260810 23:02:29.578480 8092 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35105
I20260810 23:02:29.578603 8092 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:29.578825 8092 heartbeater.cc:511] Master 127.6.47.190:35105 requested a full tablet report, sending...
I20260810 23:02:29.579229 7463 ts_manager.cc:194] Registered new tserver with Master: ef4e7cc1b9504bfc900c0d6f68ee8993 (127.6.47.130:41045)
I20260810 23:02:29.579593 7463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.130:45243
W20260810 23:02:29.660113 8097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:29.660311 8097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:29.660333 8097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:29.661553 8097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:29.661594 8097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:29.661659 8097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:29.663112 8097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:36847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:35105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:29.663355 8097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:29.663684 8097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:29.668912 8115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.668951 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:29.669116 8112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:29.669324 8097 server_base.cc:1034] running on GCE node
I20260810 23:02:29.669559 8097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:29.669883 8097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:29.671056 8097 hybrid_clock.cc:648] HybridClock initialized: now 1786402949671043 us; error 31 us; skew 500 ppm
I20260810 23:02:29.672367 8097 webserver.cc:460] Webserver started at http://127.6.47.131:39947/ using document root <none> and password file <none>
I20260810 23:02:29.672652 8097 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:29.672701 8097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:29.672853 8097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:29.673998 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "43a88ee9563344ee917048630b60e60d"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "cc7f47dc58f2551950225bbcd7a03d0e"
server_key_iv: "46b22b81b9721025e7952a7c4c1f3767"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.674386 8097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "43a88ee9563344ee917048630b60e60d"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "cc7f47dc58f2551950225bbcd7a03d0e"
server_key_iv: "46b22b81b9721025e7952a7c4c1f3767"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.675663 8097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:29.676360 8127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.676569 8097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:29.676651 8097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "43a88ee9563344ee917048630b60e60d"
format_stamp: "Formatted at 2026-08-10 23:02:29 on dist-test-slave-rwrg"
server_key: "cc7f47dc58f2551950225bbcd7a03d0e"
server_key_iv: "46b22b81b9721025e7952a7c4c1f3767"
server_key_version: "encryptionkey@0"
I20260810 23:02:29.676743 8097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:29.688656 8097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:29.688961 8097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:29.689112 8097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:29.689399 8097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:29.689759 8097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:29.689795 8097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.689846 8097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:29.689877 8097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:29.696074 8097 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:46433
I20260810 23:02:29.696134 8326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:46433 every 8 connection(s)
I20260810 23:02:29.696449 8097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:29.701514 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.697273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:38323 (local address 127.6.47.190:35105)
0810 23:02:29.697342 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.697344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.697580 (+ 236us) server_negotiation.cc:407] Connection header received
0810 23:02:29.698057 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.698058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.698068 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.698126 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:29.698774 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.699241 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.700238 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.700453 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.700556 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.701283 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:29.701284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:29.701284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:29.701289 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:29.701301 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:29.701305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:29.701347 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.701429 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.701455 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 23:02:29.701722 8329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.697234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:35105 (local address 127.6.47.131:38323)
0810 23:02:29.697450 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.697492 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.697976 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.698181 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.698183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.698192 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:29.698704 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.698709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.699336 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.699337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.700146 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.700149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.700556 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.700557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.700677 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.701152 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:29.701152 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:29.701180 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:29.701202 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:29.701361 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:29.701365 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:29.701368 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.701418 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.701497 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260810 23:02:29.702297 8327 heartbeater.cc:347] Connected to a master server at 127.6.47.190:35105
I20260810 23:02:29.702395 8327 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:29.702474 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8097
I20260810 23:02:29.702564 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:29.702608 8327 heartbeater.cc:511] Master 127.6.47.190:35105 requested a full tablet report, sending...
I20260810 23:02:29.702751 6334 external_mini_cluster.cc:1428] Setting key e6556df672d87f337a087196fd8a1724
I20260810 23:02:29.703138 7463 ts_manager.cc:194] Registered new tserver with Master: 43a88ee9563344ee917048630b60e60d (127.6.47.131:46433)
I20260810 23:02:29.703605 7463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.131:38323
I20260810 23:02:29.703846 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:29.708401 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.705325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.6.47.190:35105)
0810 23:02:29.705398 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.705400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.705477 (+ 77us) server_negotiation.cc:407] Connection header received
0810 23:02:29.705557 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.705557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.705569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.705635 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:29.705822 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.706308 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.707090 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.707285 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.707360 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.708092 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:29.708094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:29.708094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:29.708099 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:29.708119 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:29.708123 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:29.708232 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.708313 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.708324 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 23:02:29.710244 7463 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:29.712153 7463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:29.719648 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.715562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.6.47.129:43107)
0810 23:02:29.715951 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.715956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.715967 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:02:29.716009 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.716011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.716025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.716271 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.716323 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717488 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.718755 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719229 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.719314 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.719405 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.719523 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.719560 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":99,"threads_started":1}
I20260810 23:02:29.719655 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.714851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:43107 (local address 127.0.0.1:43516)
0810 23:02:29.715194 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.715206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.715261 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.716141 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.716142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.716144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.716184 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.716185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717582 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.717583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.718664 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.718666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719330 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.719331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.719422 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.719436 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.719508 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.719562 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260810 23:02:29.719666 8340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.714822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46433 (local address 127.0.0.1:39710)
0810 23:02:29.715013 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.715070 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.715138 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.715750 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.715754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.715757 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.715838 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.715840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717174 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.717176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.718551 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.718554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719427 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.719428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.719490 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.719497 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.719528 (+ 31us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.719573 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":71,"thread_start_us":59,"threads_started":1}
I20260810 23:02:29.719764 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.715146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46606 (local address 127.6.47.130:41045)
0810 23:02:29.715746 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.715751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.715821 (+ 70us) server_negotiation.cc:407] Connection header received
0810 23:02:29.715911 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.715913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.715929 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.715999 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.716207 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717057 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.718488 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719252 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.719323 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.719414 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.719616 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.719656 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":103,"threads_started":1}
I20260810 23:02:29.720212 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.715032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:41045 (local address 127.0.0.1:46606)
0810 23:02:29.715765 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.715775 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.715831 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.716068 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.716070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.716072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.716113 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.716114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717149 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.717150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.718394 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.718397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719444 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.719444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.719490 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.719497 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.719546 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.719577 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":80,"thread_start_us":97,"threads_started":1}
I20260810 23:02:29.720451 7722 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 (DEFAULT_TABLE table=test-table [id=2d3838a53c394b0b8ef4fc0bef1a0443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:29.720654 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.715350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.6.47.131:46433)
0810 23:02:29.715534 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.715539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.715551 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:02:29.715592 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.715594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.715624 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.715921 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.715980 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.717073 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.718640 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.719334 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.720367 (+ 1033us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.720473 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.720497 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.720567 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
I20260810 23:02:29.720944 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1849474964b5bba7b37ea53f04fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:29.721380 8214 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 (DEFAULT_TABLE table=test-table [id=2d3838a53c394b0b8ef4fc0bef1a0443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:29.721709 8214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1849474964b5bba7b37ea53f04fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:29.723838 8355 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: Bootstrap starting.
I20260810 23:02:29.724184 7968 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 (DEFAULT_TABLE table=test-table [id=2d3838a53c394b0b8ef4fc0bef1a0443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:29.724607 8355 tablet_bootstrap.cc:654] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:29.724615 7968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1849474964b5bba7b37ea53f04fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:29.724679 8356 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Bootstrap starting.
I20260810 23:02:29.724924 8355 log.cc:824] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:29.725333 8356 tablet_bootstrap.cc:654] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:29.725648 8355 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: No bootstrap required, opened a new log
I20260810 23:02:29.725661 8356 log.cc:824] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:29.725733 8355 ts_tablet_manager.cc:1397] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:02:29.726459 8356 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: No bootstrap required, opened a new log
I20260810 23:02:29.726534 8356 ts_tablet_manager.cc:1397] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 23:02:29.728120 8364 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Bootstrap starting.
I20260810 23:02:29.728060 8355 raft_consensus.cc:348] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.728255 8355 raft_consensus.cc:374] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:29.728302 8355 raft_consensus.cc:729] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5180b4197e540d7a10f836ba60c3752, State: Initialized, Role: FOLLOWER
I20260810 23:02:29.728461 8355 consensus_queue.cc:260] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.728794 8355 ts_tablet_manager.cc:1428] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 23:02:29.728899 8364 tablet_bootstrap.cc:654] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:29.728931 7838 heartbeater.cc:503] Master 127.6.47.190:35105 was elected leader, sending a full tablet report...
I20260810 23:02:29.729219 8364 log.cc:824] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:29.730010 8364 tablet_bootstrap.cc:492] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: No bootstrap required, opened a new log
I20260810 23:02:29.729935 8356 raft_consensus.cc:348] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.730065 8364 ts_tablet_manager.cc:1397] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:02:29.730073 8356 raft_consensus.cc:374] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:29.730131 8356 raft_consensus.cc:729] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a88ee9563344ee917048630b60e60d, State: Initialized, Role: FOLLOWER
I20260810 23:02:29.730249 8356 consensus_queue.cc:260] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.730541 8327 heartbeater.cc:503] Master 127.6.47.190:35105 was elected leader, sending a full tablet report...
I20260810 23:02:29.730754 8356 ts_tablet_manager.cc:1428] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 23:02:29.732625 8364 raft_consensus.cc:348] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.732782 8364 raft_consensus.cc:374] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:29.732829 8364 raft_consensus.cc:729] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef4e7cc1b9504bfc900c0d6f68ee8993, State: Initialized, Role: FOLLOWER
I20260810 23:02:29.732939 8364 consensus_queue.cc:260] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.733196 8364 ts_tablet_manager.cc:1428] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 23:02:29.733268 8092 heartbeater.cc:503] Master 127.6.47.190:35105 was elected leader, sending a full tablet report...
W20260810 23:02:29.822561 8093 tablet.cc:2369] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:02:29.871749 8371 raft_consensus.cc:482] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:29.871873 8371 raft_consensus.cc:504] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.872218 8371 leader_election.cc:288] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43a88ee9563344ee917048630b60e60d (127.6.47.131:46433), b5180b4197e540d7a10f836ba60c3752 (127.6.47.129:43107)
I20260810 23:02:29.875161 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.872538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:51191 (local address 127.6.47.129:43107)
0810 23:02:29.872597 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.872599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.872749 (+ 150us) server_negotiation.cc:407] Connection header received
0810 23:02:29.872777 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.872777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.872789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.872849 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.873023 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.873427 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874440 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.874801 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874892 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.874900 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.875050 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.875088 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:02:29.875162 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.872472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:41047 (local address 127.6.47.131:46433)
0810 23:02:29.872545 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.872547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.872559 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:02:29.872664 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.872665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.872675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.872740 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:29.872915 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.873409 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874439 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.874796 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874851 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.874880 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.875049 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.875088 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:02:29.875183 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.872395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:46433 (local address 127.6.47.130:41047)
0810 23:02:29.872513 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.872526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.872585 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.872765 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.872767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.872768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.872842 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.872844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.873516 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.873517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.874333 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.874335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.874882 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.874963 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.874967 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.875041 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.875088 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":44}
I20260810 23:02:29.875185 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.872443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:43107 (local address 127.6.47.130:51191)
0810 23:02:29.872651 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.872661 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:29.872733 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:29.872858 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:29.872859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:29.872860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:29.872950 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.872951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.873548 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.873549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.874331 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:29.874333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.874924 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.874925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:29.874971 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.874974 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:29.875041 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:02:29.875083 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":141,"threads_started":1}
I20260810 23:02:29.875558 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1849474964b5bba7b37ea53f04fc7" candidate_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a88ee9563344ee917048630b60e60d" is_pre_election: true
I20260810 23:02:29.875720 8254 raft_consensus.cc:2454] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4e7cc1b9504bfc900c0d6f68ee8993 in term 0.
I20260810 23:02:29.875799 7760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1849474964b5bba7b37ea53f04fc7" candidate_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5180b4197e540d7a10f836ba60c3752" is_pre_election: true
I20260810 23:02:29.875969 7760 raft_consensus.cc:2454] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef4e7cc1b9504bfc900c0d6f68ee8993 in term 0.
I20260810 23:02:29.875996 7876 leader_election.cc:302] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a88ee9563344ee917048630b60e60d, ef4e7cc1b9504bfc900c0d6f68ee8993; no voters:
I20260810 23:02:29.876119 8371 raft_consensus.cc:2781] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:29.876178 8371 raft_consensus.cc:482] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:29.876204 8371 raft_consensus.cc:3037] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.876915 8371 raft_consensus.cc:504] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.877075 8371 leader_election.cc:288] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [CANDIDATE]: Term 1 election: Requested vote from peers 43a88ee9563344ee917048630b60e60d (127.6.47.131:46433), b5180b4197e540d7a10f836ba60c3752 (127.6.47.129:43107)
I20260810 23:02:29.877250 8254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1849474964b5bba7b37ea53f04fc7" candidate_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a88ee9563344ee917048630b60e60d"
I20260810 23:02:29.877267 7760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1849474964b5bba7b37ea53f04fc7" candidate_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5180b4197e540d7a10f836ba60c3752"
I20260810 23:02:29.877362 8254 raft_consensus.cc:3037] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.877372 7760 raft_consensus.cc:3037] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:29.878125 8254 raft_consensus.cc:2454] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4e7cc1b9504bfc900c0d6f68ee8993 in term 1.
I20260810 23:02:29.878134 7760 raft_consensus.cc:2454] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef4e7cc1b9504bfc900c0d6f68ee8993 in term 1.
I20260810 23:02:29.878334 7876 leader_election.cc:302] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a88ee9563344ee917048630b60e60d, ef4e7cc1b9504bfc900c0d6f68ee8993; no voters:
I20260810 23:02:29.878430 8371 raft_consensus.cc:2781] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:29.878597 8371 raft_consensus.cc:686] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 1 LEADER]: Becoming Leader. State: Replica: ef4e7cc1b9504bfc900c0d6f68ee8993, State: Running, Role: LEADER
I20260810 23:02:29.878698 8371 consensus_queue.cc:237] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } }
I20260810 23:02:29.879628 7463 catalog_manager.cc:5495] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef4e7cc1b9504bfc900c0d6f68ee8993 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef4e7cc1b9504bfc900c0d6f68ee8993" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 41045 } health_report { overall_health: HEALTHY } } }
I20260810 23:02:29.901371 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.898770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.6.47.130:41045)
0810 23:02:29.898826 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:29.898828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:29.898837 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:02:29.898928 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:29.898928 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:29.898952 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:29.899028 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:29.899182 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.899631 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.900430 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:29.900722 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:29.900796 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:29.900970 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:29.901078 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:29.901188 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:29.901252 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 23:02:29.901261 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 23:02:29.903286 8254 raft_consensus.cc:1261] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 1 FOLLOWER]: Refusing update from remote peer ef4e7cc1b9504bfc900c0d6f68ee8993: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:29.903321 7760 raft_consensus.cc:1261] T e8c1849474964b5bba7b37ea53f04fc7 P b5180b4197e540d7a10f836ba60c3752 [term 1 FOLLOWER]: Refusing update from remote peer ef4e7cc1b9504bfc900c0d6f68ee8993: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:29.903656 8374 consensus_queue.cc:1035] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43a88ee9563344ee917048630b60e60d" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:29.903805 8371 consensus_queue.cc:1035] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5180b4197e540d7a10f836ba60c3752" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 43107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:29.906730 8382 mvcc.cc:205] Tried to move back new op lower bound from 7317106482800758784 to 7317106482703847424. Current Snapshot: MvccSnapshot[applied={T|T < 7317106482800758784}]
I20260810 23:02:29.909423 7463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:29.909518 7463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:29.910338 7463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e23f651b756474eadc10c4ea4304e1c: Sending DeleteTablet for 3 replicas of tablet e8c1849474964b5bba7b37ea53f04fc7
I20260810 23:02:29.910660 8214 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='slave'} at 127.0.0.1:39710
I20260810 23:02:29.910743 7722 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='slave'} at 127.0.0.1:43516
I20260810 23:02:29.910882 7968 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1849474964b5bba7b37ea53f04fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:29 UTC) from {username='slave'} at 127.0.0.1:46606
I20260810 23:02:29.910948 8399 tablet_replica.cc:330] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: stopping tablet replica
I20260810 23:02:29.911052 8399 raft_consensus.cc:2229] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:29.911176 8399 raft_consensus.cc:2258] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:29.911187 8402 tablet_replica.cc:330] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: stopping tablet replica
I20260810 23:02:29.911386 8402 raft_consensus.cc:2229] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:29.911545 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7581
I20260810 23:02:29.911561 8402 raft_consensus.cc:2258] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:29.912096 8399 ts_tablet_manager.cc:1905] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:29.914376 8399 ts_tablet_manager.cc:1918] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:29.914458 8399 log.cc:1196] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/e8c1849474964b5bba7b37ea53f04fc7
I20260810 23:02:29.914773 8399 ts_tablet_manager.cc:1939] T e8c1849474964b5bba7b37ea53f04fc7 P 43a88ee9563344ee917048630b60e60d: Deleting consensus metadata
I20260810 23:02:29.915655 8402 ts_tablet_manager.cc:1905] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:29.917884 8402 ts_tablet_manager.cc:1918] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:29.917950 8402 log.cc:1196] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/e8c1849474964b5bba7b37ea53f04fc7
I20260810 23:02:29.918165 8402 ts_tablet_manager.cc:1939] T e8c1849474964b5bba7b37ea53f04fc7 P ef4e7cc1b9504bfc900c0d6f68ee8993: Deleting consensus metadata
I20260810 23:02:29.930805 7442 catalog_manager.cc:4841] TS 43a88ee9563344ee917048630b60e60d (127.6.47.131:46433): tablet e8c1849474964b5bba7b37ea53f04fc7 (table test-table [id=2d3838a53c394b0b8ef4fc0bef1a0443]) successfully deleted
I20260810 23:02:29.935796 7442 catalog_manager.cc:4841] TS ef4e7cc1b9504bfc900c0d6f68ee8993 (127.6.47.130:41045): tablet e8c1849474964b5bba7b37ea53f04fc7 (table test-table [id=2d3838a53c394b0b8ef4fc0bef1a0443]) successfully deleted
W20260810 23:02:29.943506 7435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:43107 (error 108)
I20260810 23:02:29.943688 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7844
I20260810 23:02:29.948179 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:29.948039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:43107 (local address 127.0.0.1:43530)
0810 23:02:29.948084 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:29.948128 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:43107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260810 23:02:29.948410 7435 catalog_manager.cc:4584] TS b5180b4197e540d7a10f836ba60c3752 (127.6.47.129:43107): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8c1849474964b5bba7b37ea53f04fc7: Network error: Client connection negotiation failed: client connection to 127.6.47.129:43107: connect: Connection refused (error 111)
I20260810 23:02:29.954000 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8097
I20260810 23:02:29.969447 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 7408
2026-08-10T23:02:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (850 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:30 dist-test-slave-rwrg krb5kdc[8408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:30 dist-test-slave-rwrg krb5kdc[8408](info): set up 2 sockets
Aug 10 23:02:30 dist-test-slave-rwrg krb5kdc[8408](info): commencing operation
krb5kdc: starting...
W20260810 23:02:31.253681 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:31.377656 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39919
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:39919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:31.459966 8433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:31.460168 8433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:31.460191 8433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:31.461239 8433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:31.461277 8433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:31.461292 8433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:31.461303 8433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:31.461315 8433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:31.462809 8433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:39919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:31.463056 8433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:31.463224 8433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:31.467926 8439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.467957 8447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:31.468004 8433 server_base.cc:1034] running on GCE node
W20260810 23:02:31.467964 8444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:31.468381 8433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:31.468737 8433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:31.469882 8433 hybrid_clock.cc:648] HybridClock initialized: now 1786402951469862 us; error 33 us; skew 500 ppm
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:31.472402 8433 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:31.472904 8433 webserver.cc:460] Webserver started at http://127.6.47.190:39007/ using document root <none> and password file <none>
I20260810 23:02:31.473127 8433 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:31.473179 8433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:31.473333 8433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:31.474406 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "2c18c006ce554f8f97dc034d3fe42fdc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "118243211829bd1fe402d5fff0c69493"
server_key_iv: "cb8b77db6adebe33866b04e86f38c25b"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.474776 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "2c18c006ce554f8f97dc034d3fe42fdc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "118243211829bd1fe402d5fff0c69493"
server_key_iv: "cb8b77db6adebe33866b04e86f38c25b"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.476060 8433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:31.476816 8463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.476982 8433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:31.477087 8433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "2c18c006ce554f8f97dc034d3fe42fdc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "118243211829bd1fe402d5fff0c69493"
server_key_iv: "cb8b77db6adebe33866b04e86f38c25b"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.477180 8433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:31.497792 8433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:31.499634 8433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:31.499835 8433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:31.503937 8433 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:39919
I20260810 23:02:31.504010 8565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:39919 every 8 connection(s)
I20260810 23:02:31.504367 8433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:31.505307 8566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:31.507556 8566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: Bootstrap starting.
I20260810 23:02:31.508113 8566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:31.508363 8566 log.cc:824] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:31.509070 8566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: No bootstrap required, opened a new log
I20260810 23:02:31.510663 8566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } }
I20260810 23:02:31.510802 8566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:31.510824 8566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c18c006ce554f8f97dc034d3fe42fdc, State: Initialized, Role: FOLLOWER
I20260810 23:02:31.510967 8566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } }
I20260810 23:02:31.511054 8566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:31.511080 8566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:31.511108 8566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:31.511610 8566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } }
I20260810 23:02:31.511698 8566 leader_election.cc:302] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c18c006ce554f8f97dc034d3fe42fdc; no voters:
I20260810 23:02:31.511831 8566 leader_election.cc:288] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:31.511899 8573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:31.512073 8573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [term 1 LEADER]: Becoming Leader. State: Replica: 2c18c006ce554f8f97dc034d3fe42fdc, State: Running, Role: LEADER
I20260810 23:02:31.512109 8566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:31.512181 8573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } }
I20260810 23:02:31.512575 8575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c18c006ce554f8f97dc034d3fe42fdc. Latest consensus state: current_term: 1 leader_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } } }
I20260810 23:02:31.512682 8575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:31.512704 8574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c18c006ce554f8f97dc034d3fe42fdc" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39919 } } }
I20260810 23:02:31.512765 8574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:31.512900 8592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:31.513310 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8433
I20260810 23:02:31.513432 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:31.513615 6334 external_mini_cluster.cc:1428] Setting key 3ba8690b32039735ce28ffd5daecbeb9
I20260810 23:02:31.514158 8592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:31.515853 8592 catalog_manager.cc:1347] Generated new cluster ID: 4c2035edbaac4befbb32cf2765cfb88c
I20260810 23:02:31.515913 8592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:31.523433 8592 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:31.523957 8592 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:31.526557 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.514338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.6.47.190:39919)
0810 23:02:31.516692 (+ 2354us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:31.516699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:31.516725 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:02:31.516822 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:31.516828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:31.516874 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:31.517015 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:31.517694 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.518571 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.519322 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.519731 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.519876 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.522317 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:31.522321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:31.522321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:31.522333 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:31.524408 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.524852 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.524855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.524855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.524961 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.525177 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.525178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.525179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.525453 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:31.525616 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:31.526162 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 23:02:31.526312 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2307,"thread_start_us":83,"threads_started":1}
I20260810 23:02:31.526857 8592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: Generated new TSK 0
I20260810 23:02:31.527015 8592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
Entry for principal kudu/127.6.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab.
I20260810 23:02:31.554867 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:31.636128 8619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:31.636335 8619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:31.636356 8619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:31.637492 8619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:31.637540 8619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:31.637567 8619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:31.638998 8619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.129.keytab
--principal=kudu/127.6.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:31.639323 8619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:31.639568 8619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:31.644603 8639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.644613 8630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.644623 8632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:31.644776 8619 server_base.cc:1034] running on GCE node
I20260810 23:02:31.644990 8619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:31.645267 8619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:31.646409 8619 hybrid_clock.cc:648] HybridClock initialized: now 1786402951646399 us; error 27 us; skew 500 ppm
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:31.648928 8619 init.cc:377] Logged in from keytab as kudu/127.6.47.129@KRBTEST.COM (short username kudu)
I20260810 23:02:31.649391 8619 webserver.cc:460] Webserver started at http://127.6.47.129:33305/ using document root <none> and password file <none>
I20260810 23:02:31.649631 8619 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:31.649675 8619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:31.649772 8619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:31.650763 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "715e4a32646a4d78bee7cb174a3143fc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "aeac6a43172ef76a1ffe37255dbc38d6"
server_key_iv: "b8cf74dc5f833511022305297f9da1b1"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.651129 8619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "715e4a32646a4d78bee7cb174a3143fc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "aeac6a43172ef76a1ffe37255dbc38d6"
server_key_iv: "b8cf74dc5f833511022305297f9da1b1"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.652468 8619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:31.653204 8654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.653391 8619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:31.653482 8619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "715e4a32646a4d78bee7cb174a3143fc"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "aeac6a43172ef76a1ffe37255dbc38d6"
server_key_iv: "b8cf74dc5f833511022305297f9da1b1"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.653558 8619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:31.660211 8619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:31.661998 8619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:31.662159 8619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:31.662370 8619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:31.662665 8619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:31.662698 8619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.662719 8619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:31.662760 8619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.668692 8619 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:45289
I20260810 23:02:31.668762 8883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:45289 every 8 connection(s)
I20260810 23:02:31.669080 8619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:31.669860 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8619
I20260810 23:02:31.669965 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:31.670187 6334 external_mini_cluster.cc:1428] Setting key 848640693d04dd4035d41d0f779612fc
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.129@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.130@KRBTEST.COM; defaulting to no policy
I20260810 23:02:31.680742 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.669887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:59773 (local address 127.6.47.190:39919)
0810 23:02:31.669958 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:31.669962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:31.670242 (+ 280us) server_negotiation.cc:407] Connection header received
0810 23:02:31.670956 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:31.670956 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:31.670966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:31.671016 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:31.672377 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.672964 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.674117 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.674329 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.674424 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.677476 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:31.677478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:31.677478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:31.677486 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:31.679131 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.679642 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.679643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.679643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.679710 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.679990 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.679991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.679991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.680119 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:31.680196 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:31.680545 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 23:02:31.680648 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 23:02:31.680882 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.669858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39919 (local address 127.6.47.129:59773)
0810 23:02:31.670081 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:31.670134 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:31.670759 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:31.671121 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:31.671125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:31.671995 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:31.672273 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.672279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.673094 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.673095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.673986 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.673988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.674449 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.674450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.674590 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.675116 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:31.675124 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:31.677354 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:31.679236 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.679244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.679246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.679554 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.679785 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.679786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.679786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.679905 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.680187 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:31.680195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:31.680350 (+ 155us) client_negotiation.cc:769] Sending connection context
0810 23:02:31.680463 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 23:02:31.680653 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
I20260810 23:02:31.681386 8884 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39919
I20260810 23:02:31.681504 8884 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:31.681700 8884 heartbeater.cc:511] Master 127.6.47.190:39919 requested a full tablet report, sending...
I20260810 23:02:31.682330 8498 ts_manager.cc:194] Registered new tserver with Master: 715e4a32646a4d78bee7cb174a3143fc (127.6.47.129:45289)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.130@KRBTEST.COM" created.
I20260810 23:02:31.683117 8498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.129@KRBTEST.COM'} at 127.6.47.129:59773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
Entry for principal kudu/127.6.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab.
I20260810 23:02:31.698675 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:31.782394 8900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:31.782588 8900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:31.782612 8900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:31.783723 8900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:31.783763 8900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:31.783790 8900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:31.785151 8900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.130.keytab
--principal=kudu/127.6.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:31.785351 8900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:31.785526 8900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:02:31.790596 8900 server_base.cc:1034] running on GCE node
W20260810 23:02:31.790477 8909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.790508 8914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.790491 8907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:31.790920 8900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:31.791119 8900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:31.792263 8900 hybrid_clock.cc:648] HybridClock initialized: now 1786402951792242 us; error 36 us; skew 500 ppm
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:31.794823 8900 init.cc:377] Logged in from keytab as kudu/127.6.47.130@KRBTEST.COM (short username kudu)
I20260810 23:02:31.795316 8900 webserver.cc:460] Webserver started at http://127.6.47.130:38375/ using document root <none> and password file <none>
I20260810 23:02:31.795563 8900 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:31.795634 8900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:31.795782 8900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:31.796905 8900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "47f5d9e0e9094936a7ea22367c9176d1"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "2cc300feae7b84e4329cc8dedd36fa83"
server_key_iv: "59b763d199e522aa5935de3ec2193395"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.797284 8900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "47f5d9e0e9094936a7ea22367c9176d1"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "2cc300feae7b84e4329cc8dedd36fa83"
server_key_iv: "59b763d199e522aa5935de3ec2193395"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.798610 8900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:31.799367 8928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.799548 8900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:31.799634 8900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "47f5d9e0e9094936a7ea22367c9176d1"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "2cc300feae7b84e4329cc8dedd36fa83"
server_key_iv: "59b763d199e522aa5935de3ec2193395"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.799716 8900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:31.806399 8900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:31.808104 8900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:31.808269 8900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:31.808542 8900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:31.808876 8900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:31.808952 8900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.808995 8900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:31.809021 8900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.815333 8900 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:34239
I20260810 23:02:31.815376 9121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:34239 every 8 connection(s)
I20260810 23:02:31.815719 8900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.130@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:31.823941 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 8900
I20260810 23:02:31.824065 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:31.824268 6334 external_mini_cluster.cc:1428] Setting key 06e92ad48451aece18b6e2f4f71cd0a9
I20260810 23:02:31.825129 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.816480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:56813 (local address 127.6.47.190:39919)
0810 23:02:31.816544 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:31.816546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:31.816765 (+ 219us) server_negotiation.cc:407] Connection header received
0810 23:02:31.817307 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:31.817309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:31.817322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:31.817387 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:31.818341 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.818854 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.819888 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.820088 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.820180 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.822330 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:31.822332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:31.822332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:31.822340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:31.823513 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.823905 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.823907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.823907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.823970 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.824347 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.824348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.824349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.824494 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:31.824582 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:31.824942 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 23:02:31.825033 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:02:31.825356 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.816429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39919 (local address 127.6.47.130:56813)
0810 23:02:31.816639 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:31.816675 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:31.817177 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:31.817459 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:31.817463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:31.818038 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:31.818232 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.818237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.818957 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.818958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.819791 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.819794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.820190 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.820191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.820317 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.820807 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:31.820815 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:31.822189 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:31.823600 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.823607 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.823610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.823826 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.824075 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.824076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.824077 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.824243 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.824630 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:31.824639 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:31.824806 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 23:02:31.824895 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 23:02:31.825091 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260810 23:02:31.825788 9122 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39919
I20260810 23:02:31.825874 9122 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:31.826072 9122 heartbeater.cc:511] Master 127.6.47.190:39919 requested a full tablet report, sending...
I20260810 23:02:31.826481 8498 ts_manager.cc:194] Registered new tserver with Master: 47f5d9e0e9094936a7ea22367c9176d1 (127.6.47.130:34239)
I20260810 23:02:31.826975 8498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.130@KRBTEST.COM'} at 127.6.47.130:56813
WARNING: no policy specified for kudu/127.6.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
Entry for principal kudu/127.6.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab.
I20260810 23:02:31.851089 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:38595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:31.934767 9130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:31.934998 9130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:31.935039 9130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:31.936273 9130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:31.936338 9130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:31.936383 9130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:31.937898 9130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:38595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.131.keytab
--principal=kudu/127.6.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:39919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:31.938169 9130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:31.938400 9130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:31.943398 9143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.943500 9148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:31.943567 9138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:31.943842 9130 server_base.cc:1034] running on GCE node
I20260810 23:02:31.944061 9130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:31.944329 9130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:31.945494 9130 hybrid_clock.cc:648] HybridClock initialized: now 1786402951945490 us; error 40 us; skew 500 ppm
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:31.948019 9130 init.cc:377] Logged in from keytab as kudu/127.6.47.131@KRBTEST.COM (short username kudu)
I20260810 23:02:31.948484 9130 webserver.cc:460] Webserver started at http://127.6.47.131:46235/ using document root <none> and password file <none>
I20260810 23:02:31.948738 9130 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:31.948779 9130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:31.948868 9130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:31.949885 9130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "ef8e738f0ad842b083b50ab554032cc4"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "be21e27acbdd9c6c096d96bc618e9b19"
server_key_iv: "4350412e9c0322ceb1b6088143b1be04"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.950206 9130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "ef8e738f0ad842b083b50ab554032cc4"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "be21e27acbdd9c6c096d96bc618e9b19"
server_key_iv: "4350412e9c0322ceb1b6088143b1be04"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.951493 9130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:31.952131 9162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.952317 9130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:31.952414 9130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "ef8e738f0ad842b083b50ab554032cc4"
format_stamp: "Formatted at 2026-08-10 23:02:31 on dist-test-slave-rwrg"
server_key: "be21e27acbdd9c6c096d96bc618e9b19"
server_key_iv: "4350412e9c0322ceb1b6088143b1be04"
server_key_version: "encryptionkey@0"
I20260810 23:02:31.952522 9130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:31.965665 9130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:31.967552 9130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:31.967713 9130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:31.967954 9130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:31.968286 9130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:31.968320 9130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.968341 9130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:31.968389 9130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:31.974539 9130 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:45395
I20260810 23:02:31.974637 9343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:45395 every 8 connection(s)
I20260810 23:02:31.974967 9130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:31.976471 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9130
I20260810 23:02:31.976612 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:31.976836 6334 external_mini_cluster.cc:1428] Setting key 940bc850e1f7b6462347bc964ba4b133
Aug 10 23:02:31 dist-test-slave-rwrg krb5kdc[8408](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402951, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.131@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:31.986157 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.975773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:36323 (local address 127.6.47.190:39919)
0810 23:02:31.975857 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:31.975860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:31.976051 (+ 191us) server_negotiation.cc:407] Connection header received
0810 23:02:31.976904 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:31.976906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:31.976919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:31.976983 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:31.978198 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.978676 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.979905 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.980160 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.980230 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.982578 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:31.982580 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:31.982580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:31.982587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:31.984293 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.984901 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.984903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.984903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.985001 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:31.985337 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:31.985338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:31.985338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:31.985485 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:31.985574 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:31.985941 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 23:02:31.986041 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:02:31.986304 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:31.975687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:39919 (local address 127.6.47.131:36323)
0810 23:02:31.975917 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:31.975965 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:31.976769 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:31.977063 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:31.977071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:31.977894 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:31.978112 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.978117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.978759 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.978760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.979822 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:31.979826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:31.980244 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:31.980246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:31.980397 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:31.980990 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:02:31.981000 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:31.982472 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:31.984396 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.984403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.984405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.984797 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.985085 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:02:31.985087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:02:31.985087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:02:31.985260 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:02:31.985585 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:31.985592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:31.985760 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 23:02:31.985874 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 23:02:31.986042 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260810 23:02:31.989491 9344 heartbeater.cc:347] Connected to a master server at 127.6.47.190:39919
I20260810 23:02:31.989599 9344 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:31.989815 9344 heartbeater.cc:511] Master 127.6.47.190:39919 requested a full tablet report, sending...
I20260810 23:02:31.990271 8498 ts_manager.cc:194] Registered new tserver with Master: ef8e738f0ad842b083b50ab554032cc4 (127.6.47.131:45395)
I20260810 23:02:31.990747 8498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.47.131@KRBTEST.COM'} at 127.6.47.131:36323
I20260810 23:02:32.001017 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:32.008132 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.002494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.6.47.190:39919)
0810 23:02:32.002554 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.002555 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.002705 (+ 150us) server_negotiation.cc:407] Connection header received
0810 23:02:32.002758 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.002759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.002768 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.002815 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.003356 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.003871 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.004611 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.004787 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.004835 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.005937 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.005938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.005938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:32.005944 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.007127 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:32.007470 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:32.007471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:32.007472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:32.007522 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:32.007698 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:32.007698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:32.007699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:32.007836 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.007909 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.008003 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.008070 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:02:32.009958 8498 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:32.011606 8498 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:32.018213 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45289 (local address 127.0.0.1:41062)
0810 23:02:32.014502 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.014512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.014561 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.014831 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.014833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.014835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.014870 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.014871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.016065 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.016066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.017019 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.017022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.017869 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.017870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.017950 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.017965 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.018047 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.018100 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260810 23:02:32.018229 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.6.47.129:45289)
0810 23:02:32.014485 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.014490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.014552 (+ 62us) server_negotiation.cc:407] Connection header received
0810 23:02:32.014625 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.014627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.014643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.014741 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.014944 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.015965 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.017114 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.017777 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.017855 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.017915 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.018077 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.018118 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":91,"threads_started":1}
I20260810 23:02:32.018545 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:45395 (local address 127.0.0.1:33000)
0810 23:02:32.014376 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.014388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.014434 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.015009 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.015010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.015012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.015057 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.015058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.015940 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.015941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.017291 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.017294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.018295 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.018296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.018373 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.018376 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.018425 (+ 49us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.018462 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":85,"thread_start_us":50,"threads_started":1}
I20260810 23:02:32.018788 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.6.47.131:45395)
0810 23:02:32.014781 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.014784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.014794 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:32.014832 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.014834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.014849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.014920 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.015142 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.015830 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.017401 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.018141 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.018205 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.018258 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.018675 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.018717 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":114,"threads_started":1}
I20260810 23:02:32.018854 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:34239 (local address 127.0.0.1:51760)
0810 23:02:32.014138 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.014182 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.014249 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.014618 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.014622 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.014624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.014697 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.014700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.016029 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.016030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.017326 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.017328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.018250 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.018251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.018697 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.018703 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.018740 (+ 37us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.018789 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":70,"thread_start_us":52,"threads_started":1}
I20260810 23:02:32.019038 8742 tablet_service.cc:1467] Processing CreateTablet for tablet ca222ec8b23f456babe7d714a7fc0310 (DEFAULT_TABLE table=test-table [id=431efdbbf1b44bc397c099d5f457581f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.019201 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.014264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51760 (local address 127.6.47.130:34239)
0810 23:02:32.014444 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.014449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.014461 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:02:32.014506 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.014508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.014528 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.014631 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.014784 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.015926 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.017417 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.018145 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.018968 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.019035 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.019061 (+ 26us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.019114 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260810 23:02:32.019449 8742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca222ec8b23f456babe7d714a7fc0310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.019454 9238 tablet_service.cc:1467] Processing CreateTablet for tablet ca222ec8b23f456babe7d714a7fc0310 (DEFAULT_TABLE table=test-table [id=431efdbbf1b44bc397c099d5f457581f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.019799 9238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca222ec8b23f456babe7d714a7fc0310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.019800 9011 tablet_service.cc:1467] Processing CreateTablet for tablet ca222ec8b23f456babe7d714a7fc0310 (DEFAULT_TABLE table=test-table [id=431efdbbf1b44bc397c099d5f457581f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.020115 9011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca222ec8b23f456babe7d714a7fc0310. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.022704 9368 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Bootstrap starting.
I20260810 23:02:32.023065 9370 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: Bootstrap starting.
I20260810 23:02:32.023753 9370 tablet_bootstrap.cc:654] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.023753 9368 tablet_bootstrap.cc:654] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.024407 9368 log.cc:824] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.024406 9370 log.cc:824] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.024802 9371 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Bootstrap starting.
I20260810 23:02:32.025357 9370 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: No bootstrap required, opened a new log
I20260810 23:02:32.025441 9368 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: No bootstrap required, opened a new log
I20260810 23:02:32.025441 9370 ts_tablet_manager.cc:1397] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:32.025513 9371 tablet_bootstrap.cc:654] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.025521 9368 ts_tablet_manager.cc:1397] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 23:02:32.026120 9371 log.cc:824] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.026926 9371 tablet_bootstrap.cc:492] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: No bootstrap required, opened a new log
I20260810 23:02:32.026995 9371 ts_tablet_manager.cc:1397] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 23:02:32.027490 9368 raft_consensus.cc:348] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.027688 9368 raft_consensus.cc:374] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.027736 9368 raft_consensus.cc:729] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47f5d9e0e9094936a7ea22367c9176d1, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.027706 9370 raft_consensus.cc:348] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.027889 9370 raft_consensus.cc:374] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.027863 9368 consensus_queue.cc:260] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.027941 9370 raft_consensus.cc:729] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715e4a32646a4d78bee7cb174a3143fc, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.028061 9370 consensus_queue.cc:260] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.028180 9368 ts_tablet_manager.cc:1428] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:02:32.028232 9122 heartbeater.cc:503] Master 127.6.47.190:39919 was elected leader, sending a full tablet report...
I20260810 23:02:32.028339 9370 ts_tablet_manager.cc:1428] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:02:32.028452 8884 heartbeater.cc:503] Master 127.6.47.190:39919 was elected leader, sending a full tablet report...
I20260810 23:02:32.029625 9371 raft_consensus.cc:348] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.029819 9371 raft_consensus.cc:374] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.029853 9371 raft_consensus.cc:729] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8e738f0ad842b083b50ab554032cc4, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.029937 9371 consensus_queue.cc:260] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.030161 9385 raft_consensus.cc:482] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:32.030218 9371 ts_tablet_manager.cc:1428] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 23:02:32.030249 9344 heartbeater.cc:503] Master 127.6.47.190:39919 was elected leader, sending a full tablet report...
I20260810 23:02:32.030279 9385 raft_consensus.cc:504] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.030947 9385 leader_election.cc:288] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef8e738f0ad842b083b50ab554032cc4 (127.6.47.131:45395), 715e4a32646a4d78bee7cb174a3143fc (127.6.47.129:45289)
I20260810 23:02:32.034066 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.030761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:45395 (local address 127.6.47.130:52059)
0810 23:02:32.030809 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.030819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.030856 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.030996 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.030997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.030999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.031066 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.031068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.031951 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.031952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.033110 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.033112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.033808 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.033809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.033898 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.033902 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.033956 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.033999 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 23:02:32.034065 9367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.030804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:52059 (local address 127.6.47.131:45395)
0810 23:02:32.030842 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.030845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.030853 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:02:32.030920 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.030921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.030931 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.030992 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.031166 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.031874 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.033200 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.033727 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.033801 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.033809 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.033963 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.034003 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:02:32.034341 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.031118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:57961 (local address 127.6.47.129:45289)
0810 23:02:32.031154 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.031156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.031279 (+ 123us) server_negotiation.cc:407] Connection header received
0810 23:02:32.031328 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.031328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.031339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.031395 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.031539 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.032164 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.033372 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.033989 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.034054 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.034062 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.034248 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.034283 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:02:32.034359 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.031058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45289 (local address 127.6.47.130:57961)
0810 23:02:32.031201 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.031212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.031273 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.031399 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.031399 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.031401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.031476 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.031477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.032238 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.032239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.033303 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.033305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.034056 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.034057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.034144 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.034150 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.034239 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.034278 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260810 23:02:32.034536 9266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca222ec8b23f456babe7d714a7fc0310" candidate_uuid: "47f5d9e0e9094936a7ea22367c9176d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8e738f0ad842b083b50ab554032cc4" is_pre_election: true
I20260810 23:02:32.034657 9266 raft_consensus.cc:2454] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f5d9e0e9094936a7ea22367c9176d1 in term 0.
I20260810 23:02:32.034890 8933 leader_election.cc:302] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f5d9e0e9094936a7ea22367c9176d1, ef8e738f0ad842b083b50ab554032cc4; no voters:
I20260810 23:02:32.035023 8793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca222ec8b23f456babe7d714a7fc0310" candidate_uuid: "47f5d9e0e9094936a7ea22367c9176d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715e4a32646a4d78bee7cb174a3143fc" is_pre_election: true
I20260810 23:02:32.035058 9385 raft_consensus.cc:2781] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:32.035101 9385 raft_consensus.cc:482] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:32.035120 9385 raft_consensus.cc:3037] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.035164 8793 raft_consensus.cc:2454] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47f5d9e0e9094936a7ea22367c9176d1 in term 0.
I20260810 23:02:32.035889 9385 raft_consensus.cc:504] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.036037 9385 leader_election.cc:288] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [CANDIDATE]: Term 1 election: Requested vote from peers ef8e738f0ad842b083b50ab554032cc4 (127.6.47.131:45395), 715e4a32646a4d78bee7cb174a3143fc (127.6.47.129:45289)
I20260810 23:02:32.036219 9266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca222ec8b23f456babe7d714a7fc0310" candidate_uuid: "47f5d9e0e9094936a7ea22367c9176d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef8e738f0ad842b083b50ab554032cc4"
I20260810 23:02:32.036234 8793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca222ec8b23f456babe7d714a7fc0310" candidate_uuid: "47f5d9e0e9094936a7ea22367c9176d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "715e4a32646a4d78bee7cb174a3143fc"
I20260810 23:02:32.036304 8793 raft_consensus.cc:3037] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.036304 9266 raft_consensus.cc:3037] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.037205 8793 raft_consensus.cc:2454] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f5d9e0e9094936a7ea22367c9176d1 in term 1.
I20260810 23:02:32.037216 9266 raft_consensus.cc:2454] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47f5d9e0e9094936a7ea22367c9176d1 in term 1.
I20260810 23:02:32.037384 8933 leader_election.cc:302] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47f5d9e0e9094936a7ea22367c9176d1, ef8e738f0ad842b083b50ab554032cc4; no voters:
I20260810 23:02:32.037496 9385 raft_consensus.cc:2781] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:32.037636 9385 raft_consensus.cc:686] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 1 LEADER]: Becoming Leader. State: Replica: 47f5d9e0e9094936a7ea22367c9176d1, State: Running, Role: LEADER
I20260810 23:02:32.037745 9385 consensus_queue.cc:237] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } }
I20260810 23:02:32.038559 8489 catalog_manager.cc:5495] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47f5d9e0e9094936a7ea22367c9176d1 (127.6.47.130). New cstate: current_term: 1 leader_uuid: "47f5d9e0e9094936a7ea22367c9176d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47f5d9e0e9094936a7ea22367c9176d1" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 34239 } health_report { overall_health: HEALTHY } } }
I20260810 23:02:32.049983 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.047431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.6.47.130:34239)
0810 23:02:32.047483 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.047486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.047496 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:32.047548 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.047549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.047580 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.047649 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:32.047821 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.048290 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.049151 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.049361 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.049441 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.049623 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:32.049736 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:32.049834 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.049853 (+ 19us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.049878 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:02:32.051862 8793 raft_consensus.cc:1261] T ca222ec8b23f456babe7d714a7fc0310 P 715e4a32646a4d78bee7cb174a3143fc [term 1 FOLLOWER]: Refusing update from remote peer 47f5d9e0e9094936a7ea22367c9176d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:32.051868 9266 raft_consensus.cc:1261] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 1 FOLLOWER]: Refusing update from remote peer 47f5d9e0e9094936a7ea22367c9176d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:32.052237 9389 consensus_queue.cc:1035] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "715e4a32646a4d78bee7cb174a3143fc" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 45289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:32.052397 9385 consensus_queue.cc:1035] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef8e738f0ad842b083b50ab554032cc4" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:32.056282 9399 mvcc.cc:205] Tried to move back new op lower bound from 7317106491601518592 to 7317106491547381760. Current Snapshot: MvccSnapshot[applied={T|T < 7317106491601518592}]
I20260810 23:02:32.057159 8489 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:32.057257 8489 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:32.058100 8489 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c18c006ce554f8f97dc034d3fe42fdc: Sending DeleteTablet for 3 replicas of tablet ca222ec8b23f456babe7d714a7fc0310
I20260810 23:02:32.058442 9238 tablet_service.cc:1514] Processing DeleteTablet for tablet ca222ec8b23f456babe7d714a7fc0310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:32 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:33000
I20260810 23:02:32.058446 9011 tablet_service.cc:1514] Processing DeleteTablet for tablet ca222ec8b23f456babe7d714a7fc0310 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:32 UTC) from {username='kudu', principal='kudu/127.6.47.190@KRBTEST.COM'} at 127.0.0.1:51760
I20260810 23:02:32.058782 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8619
I20260810 23:02:32.078802 9413 tablet_replica.cc:330] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: stopping tablet replica
I20260810 23:02:32.079005 9413 raft_consensus.cc:2229] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:32.079159 9413 raft_consensus.cc:2258] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:32.079385 9410 tablet_replica.cc:330] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: stopping tablet replica
I20260810 23:02:32.079514 9410 raft_consensus.cc:2229] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:32.079648 9410 raft_consensus.cc:2258] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:32.080226 9413 ts_tablet_manager.cc:1905] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:32.082121 9413 ts_tablet_manager.cc:1918] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:32.082209 9413 log.cc:1196] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/ca222ec8b23f456babe7d714a7fc0310
I20260810 23:02:32.082525 9413 ts_tablet_manager.cc:1939] T ca222ec8b23f456babe7d714a7fc0310 P 47f5d9e0e9094936a7ea22367c9176d1: Deleting consensus metadata
I20260810 23:02:32.082556 9410 ts_tablet_manager.cc:1905] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:32.082998 8465 catalog_manager.cc:4841] TS 47f5d9e0e9094936a7ea22367c9176d1 (127.6.47.130:34239): tablet ca222ec8b23f456babe7d714a7fc0310 (table test-table [id=431efdbbf1b44bc397c099d5f457581f]) successfully deleted
W20260810 23:02:32.084448 8466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:45289 (error 108)
I20260810 23:02:32.084841 9410 ts_tablet_manager.cc:1918] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:32.085007 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.084853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:45289 (local address 127.0.0.1:41064)
0810 23:02:32.084922 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.084970 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:02:32.085019 9410 log.cc:1196] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/ca222ec8b23f456babe7d714a7fc0310
W20260810 23:02:32.085150 8466 catalog_manager.cc:4584] TS 715e4a32646a4d78bee7cb174a3143fc (127.6.47.129:45289): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca222ec8b23f456babe7d714a7fc0310: Network error: Client connection negotiation failed: client connection to 127.6.47.129:45289: connect: Connection refused (error 111)
I20260810 23:02:32.085492 9410 ts_tablet_manager.cc:1939] T ca222ec8b23f456babe7d714a7fc0310 P ef8e738f0ad842b083b50ab554032cc4: Deleting consensus metadata
I20260810 23:02:32.085530 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8900
I20260810 23:02:32.088872 8470 catalog_manager.cc:4841] TS ef8e738f0ad842b083b50ab554032cc4 (127.6.47.131:45395): tablet ca222ec8b23f456babe7d714a7fc0310 (table test-table [id=431efdbbf1b44bc397c099d5f457581f]) successfully deleted
I20260810 23:02:32.096041 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9130
I20260810 23:02:32.109644 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 8433
2026-08-10T23:02:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2142 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T23:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:32Z Disabled control of system clock
I20260810 23:02:32.146602 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40579
--webserver_interface=127.6.47.190
--webserver_port=0
--builtin_ntp_servers=127.6.47.148:43785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:40579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:32.228717 9419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:32.229015 9419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:32.229063 9419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:32.230225 9419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:32.230265 9419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:32.230278 9419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:32.230291 9419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:32.230340 9419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:32.232010 9419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:40579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:40579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:32.232308 9419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:32.232685 9419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:32.237691 9433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.237704 9430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.237686 9429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:32.237959 9419 server_base.cc:1034] running on GCE node
I20260810 23:02:32.238272 9419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:32.238538 9419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:32.239681 9419 hybrid_clock.cc:648] HybridClock initialized: now 1786402952239668 us; error 47 us; skew 500 ppm
I20260810 23:02:32.241025 9419 webserver.cc:460] Webserver started at http://127.6.47.190:46375/ using document root <none> and password file <none>
I20260810 23:02:32.241282 9419 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:32.241330 9419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:32.241489 9419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:32.242600 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "41003e5b4f0146299025840428fbb525"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "b6ec25c005dfe428159ffb81c45c51ec"
server_key_iv: "807f95aa8003661370efdacce3c48f23"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.242969 9419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "41003e5b4f0146299025840428fbb525"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "b6ec25c005dfe428159ffb81c45c51ec"
server_key_iv: "807f95aa8003661370efdacce3c48f23"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.244374 9419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:32.245142 9443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.245325 9419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:32.245422 9419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "41003e5b4f0146299025840428fbb525"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "b6ec25c005dfe428159ffb81c45c51ec"
server_key_iv: "807f95aa8003661370efdacce3c48f23"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.245504 9419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:32.259382 9419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:32.259704 9419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:32.259858 9419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:32.264389 9419 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:40579
I20260810 23:02:32.264421 9525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:40579 every 8 connection(s)
I20260810 23:02:32.264773 9419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:32.265765 9526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.267969 9526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: Bootstrap starting.
I20260810 23:02:32.268582 9526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.268858 9526 log.cc:824] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.269522 9526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: No bootstrap required, opened a new log
I20260810 23:02:32.271026 9526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } }
I20260810 23:02:32.271174 9526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.271198 9526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41003e5b4f0146299025840428fbb525, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.271301 9526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } }
I20260810 23:02:32.271381 9526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:32.271418 9526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:32.271490 9526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.271502 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9419
I20260810 23:02:32.271613 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:32.271826 6334 external_mini_cluster.cc:1428] Setting key 9cc60fea2ff5ce023fb5d1abee767bc6
I20260810 23:02:32.272145 9526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } }
I20260810 23:02:32.272258 9526 leader_election.cc:302] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41003e5b4f0146299025840428fbb525; no voters:
I20260810 23:02:32.272429 9526 leader_election.cc:288] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:32.272539 9530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:32.272711 9530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [term 1 LEADER]: Becoming Leader. State: Replica: 41003e5b4f0146299025840428fbb525, State: Running, Role: LEADER
I20260810 23:02:32.272728 9526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:32.272845 9530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } }
I20260810 23:02:32.273527 9539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41003e5b4f0146299025840428fbb525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } } }
I20260810 23:02:32.273633 9539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:32.274147 9542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41003e5b4f0146299025840428fbb525. Latest consensus state: current_term: 1 leader_uuid: "41003e5b4f0146299025840428fbb525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41003e5b4f0146299025840428fbb525" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 40579 } } }
I20260810 23:02:32.274227 9542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:32.274331 9563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:32.274988 9563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:32.276526 9563 catalog_manager.cc:1347] Generated new cluster ID: 5d3fbe3d224249c397b5b318b0d958b7
I20260810 23:02:32.276580 9563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:32.278681 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.272339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.6.47.190:40579)
0810 23:02:32.272644 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.272653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.272692 (+ 39us) server_negotiation.cc:407] Connection header received
0810 23:02:32.272804 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.272811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.272881 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.273070 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.273353 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.275112 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.276223 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.276866 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.277340 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.277758 (+ 418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.277762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.277762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:32.277968 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.278014 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:32.278028 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.278348 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.278383 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.278462 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":105,"threads_started":1}
I20260810 23:02:32.338799 9563 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:32.339371 9563 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:32.342619 9563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: Generated new TSK 0
I20260810 23:02:32.342832 9563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:32.347462 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.129:0
--local_ip_for_outbound_sockets=127.6.47.129
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--builtin_ntp_servers=127.6.47.148:43785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:32.433874 9574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:32.434142 9574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:32.434182 9574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:32.435453 9574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:32.435494 9574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:32.435519 9574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.129
I20260810 23:02:32.436894 9574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.47.129
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.129
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:32.437083 9574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:32.437350 9574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:32.442420 9584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.442590 9589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.442401 9583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:32.442574 9574 server_base.cc:1034] running on GCE node
I20260810 23:02:32.442906 9574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:32.443176 9574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:32.444345 9574 hybrid_clock.cc:648] HybridClock initialized: now 1786402952444332 us; error 29 us; skew 500 ppm
I20260810 23:02:32.445626 9574 webserver.cc:460] Webserver started at http://127.6.47.129:41939/ using document root <none> and password file <none>
I20260810 23:02:32.445875 9574 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:32.445945 9574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:32.446100 9574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:32.447175 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data/instance:
uuid: "b753b17294aa43e48bf6e4bebda7d202"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "601447a280ea56072b14b18fba634d2d"
server_key_iv: "b52bf743003a92aab2c555e22cb448b3"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.447544 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance:
uuid: "b753b17294aa43e48bf6e4bebda7d202"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "601447a280ea56072b14b18fba634d2d"
server_key_iv: "b52bf743003a92aab2c555e22cb448b3"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.449136 9574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:32.449900 9599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.450081 9574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:32.450186 9574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal
uuid: "b753b17294aa43e48bf6e4bebda7d202"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "601447a280ea56072b14b18fba634d2d"
server_key_iv: "b52bf743003a92aab2c555e22cb448b3"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.450273 9574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:32.467502 9574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:32.467839 9574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:32.467993 9574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:32.468240 9574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:32.468694 9574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:32.468752 9574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.468793 9574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:32.468822 9574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.474988 9574 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.129:34159
I20260810 23:02:32.475046 9798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.129:34159 every 8 connection(s)
I20260810 23:02:32.475425 9574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/data/info.pb
I20260810 23:02:32.480885 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.476198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.129:48887 (local address 127.6.47.190:40579)
0810 23:02:32.476265 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.476267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.476535 (+ 268us) server_negotiation.cc:407] Connection header received
0810 23:02:32.477125 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.477126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.477142 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.477195 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.477882 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.478392 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.479406 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.479618 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.479700 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.480525 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.480527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.480527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:32.480533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.480549 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:32.480553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.480650 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.480747 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.480800 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:02:32.481117 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.476176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40579 (local address 127.6.47.129:48887)
0810 23:02:32.476395 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.476430 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.477004 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.477270 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.477273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.477281 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:32.477784 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.477789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.478491 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.478492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.479309 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.479311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.479708 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.479710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.479844 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.480356 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:32.480357 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:32.480386 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:32.480408 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:32.480654 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:32.480662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:32.480665 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.480750 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.480870 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260810 23:02:32.481652 9801 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40579
I20260810 23:02:32.481755 9801 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:32.481935 9801 heartbeater.cc:511] Master 127.6.47.190:40579 requested a full tablet report, sending...
I20260810 23:02:32.482460 9475 ts_manager.cc:194] Registered new tserver with Master: b753b17294aa43e48bf6e4bebda7d202 (127.6.47.129:34159)
I20260810 23:02:32.483096 9475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.129:48887
I20260810 23:02:32.483793 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9574
I20260810 23:02:32.483906 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-0/wal/instance
I20260810 23:02:32.484086 6334 external_mini_cluster.cc:1428] Setting key 4a3e6d88aac07c2d013e9ba590496707
I20260810 23:02:32.485064 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.130:0
--local_ip_for_outbound_sockets=127.6.47.130
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--builtin_ntp_servers=127.6.47.148:43785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:32.568158 9805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:32.568358 9805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:32.568379 9805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:32.569561 9805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:32.569604 9805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:32.569630 9805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.130
I20260810 23:02:32.571075 9805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.47.130
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.130
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:32.571318 9805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:32.571616 9805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:32.576848 9815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.576848 9818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.576871 9810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:32.576958 9805 server_base.cc:1034] running on GCE node
I20260810 23:02:32.577333 9805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:32.577596 9805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:32.578773 9805 hybrid_clock.cc:648] HybridClock initialized: now 1786402952578775 us; error 46 us; skew 500 ppm
I20260810 23:02:32.580008 9805 webserver.cc:460] Webserver started at http://127.6.47.130:32905/ using document root <none> and password file <none>
I20260810 23:02:32.580267 9805 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:32.580338 9805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:32.580474 9805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:32.581574 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data/instance:
uuid: "fe503342529444809357008c7d87849b"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "7b0abfc19d77a6340d2c1a99094477d6"
server_key_iv: "3c8484415e17dd928184883f903992ca"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.581957 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance:
uuid: "fe503342529444809357008c7d87849b"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "7b0abfc19d77a6340d2c1a99094477d6"
server_key_iv: "3c8484415e17dd928184883f903992ca"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.583313 9805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:32.584033 9826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.584218 9805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:32.584329 9805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal
uuid: "fe503342529444809357008c7d87849b"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "7b0abfc19d77a6340d2c1a99094477d6"
server_key_iv: "3c8484415e17dd928184883f903992ca"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.584408 9805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:32.607030 9805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:32.607367 9805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:32.607527 9805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:32.607765 9805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:32.608129 9805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:32.608186 9805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.608222 9805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:32.608249 9805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.614182 9805 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.130:35187
I20260810 23:02:32.614231 10040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.130:35187 every 8 connection(s)
I20260810 23:02:32.614598 9805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/data/info.pb
I20260810 23:02:32.619562 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.615421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:49537 (local address 127.6.47.190:40579)
0810 23:02:32.615483 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.615490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.615691 (+ 201us) server_negotiation.cc:407] Connection header received
0810 23:02:32.616190 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.616191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.616200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.616256 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.616931 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.617420 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.618371 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.618552 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.618602 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.619343 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.619344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.619345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:32.619349 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.619361 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:32.619365 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.619404 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.619484 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.619506 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 23:02:32.619781 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.615374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40579 (local address 127.6.47.130:49537)
0810 23:02:32.615560 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.615601 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.616089 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.616318 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.616321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.616328 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:32.616857 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.616861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.617524 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.617524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.618293 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.618295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.618628 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.618629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.618748 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.619196 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:32.619196 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:32.619223 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:32.619244 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:32.619419 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:32.619423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:32.619425 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.619472 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.619541 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260810 23:02:32.620350 10041 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40579
I20260810 23:02:32.620451 10041 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:32.620651 10041 heartbeater.cc:511] Master 127.6.47.190:40579 requested a full tablet report, sending...
I20260810 23:02:32.621006 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 9805
I20260810 23:02:32.621140 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal/instance
I20260810 23:02:32.621105 9475 ts_manager.cc:194] Registered new tserver with Master: fe503342529444809357008c7d87849b (127.6.47.130:35187)
I20260810 23:02:32.621356 6334 external_mini_cluster.cc:1428] Setting key 512095ebb75d8c1e270630b3236e5dfc
I20260810 23:02:32.621555 9475 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.130:49537
I20260810 23:02:32.622236 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.47.131:0
--local_ip_for_outbound_sockets=127.6.47.131
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--builtin_ntp_servers=127.6.47.148:43785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 23:02:32.707233 10044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:32.707439 10044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:32.707460 10044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:32.708688 10044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:32.708730 10044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:32.708756 10044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.47.131
I20260810 23:02:32.710143 10044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:43785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.47.131
--webserver_port=0
--tserver_master_addrs=127.6.47.190:40579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.47.131
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:32.710407 10044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:32.710711 10044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:32.715682 10060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.715711 10058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:32.715708 10057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:32.716097 10044 server_base.cc:1034] running on GCE node
I20260810 23:02:32.716328 10044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:32.716595 10044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:32.717742 10044 hybrid_clock.cc:648] HybridClock initialized: now 1786402952717720 us; error 38 us; skew 500 ppm
I20260810 23:02:32.718981 10044 webserver.cc:460] Webserver started at http://127.6.47.131:44971/ using document root <none> and password file <none>
I20260810 23:02:32.719228 10044 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:32.719270 10044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:32.719421 10044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:32.720480 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data/instance:
uuid: "736bb3e799aa4932aab73a4d18a89557"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "ec38a409e8dec69899919cd4bf733717"
server_key_iv: "06345d37d6b6ab1dcae09b9ceb455298"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.720932 10044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance:
uuid: "736bb3e799aa4932aab73a4d18a89557"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "ec38a409e8dec69899919cd4bf733717"
server_key_iv: "06345d37d6b6ab1dcae09b9ceb455298"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.722204 10044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:32.722939 10068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.723140 10044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.723232 10044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal
uuid: "736bb3e799aa4932aab73a4d18a89557"
format_stamp: "Formatted at 2026-08-10 23:02:32 on dist-test-slave-rwrg"
server_key: "ec38a409e8dec69899919cd4bf733717"
server_key_iv: "06345d37d6b6ab1dcae09b9ceb455298"
server_key_version: "encryptionkey@0"
I20260810 23:02:32.723306 10044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:32.735587 10044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:32.735873 10044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:32.736019 10044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:32.736238 10044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 23:02:32.736634 10044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 23:02:32.736687 10044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.736727 10044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 23:02:32.736753 10044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:32.742749 10044 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.131:44635
I20260810 23:02:32.742830 10283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.131:44635 every 8 connection(s)
I20260810 23:02:32.743167 10044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/data/info.pb
I20260810 23:02:32.748098 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10044
I20260810 23:02:32.748215 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal/instance
I20260810 23:02:32.748425 6334 external_mini_cluster.cc:1428] Setting key c6128e23c2f4ecb2b3bbb6fe95591d3d
I20260810 23:02:32.748551 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.743893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.131:35671 (local address 127.6.47.190:40579)
0810 23:02:32.743944 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.743947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.744257 (+ 310us) server_negotiation.cc:407] Connection header received
0810 23:02:32.744819 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.744821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.744831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.744877 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.745612 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.746127 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.747136 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.747332 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.747385 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.748249 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.748251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.748251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:32.748257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.748275 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:32.748281 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.748339 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.748415 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.748448 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:02:32.748790 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.743886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:40579 (local address 127.6.47.131:35671)
0810 23:02:32.744122 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.744167 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.744727 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.744936 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.744939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.744946 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:02:32.745535 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.745541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.746222 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.746223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.747055 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.747057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.747411 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.747412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.747523 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.748066 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 23:02:32.748066 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:02:32.748094 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 23:02:32.748117 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:02:32.748341 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:02:32.748347 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:02:32.748350 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.748416 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.748549 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260810 23:02:32.749248 10284 heartbeater.cc:347] Connected to a master server at 127.6.47.190:40579
I20260810 23:02:32.749337 10284 heartbeater.cc:464] Registering TS with master...
I20260810 23:02:32.749521 10284 heartbeater.cc:511] Master 127.6.47.190:40579 requested a full tablet report, sending...
I20260810 23:02:32.749861 9473 ts_manager.cc:194] Registered new tserver with Master: 736bb3e799aa4932aab73a4d18a89557 (127.6.47.131:44635)
I20260810 23:02:32.750241 9473 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.47.131:35671
I20260810 23:02:32.760561 6334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 23:02:32.764994 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.762000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.6.47.190:40579)
0810 23:02:32.762082 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.762084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.762193 (+ 109us) server_negotiation.cc:407] Connection header received
0810 23:02:32.762292 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.762293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.762306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.762356 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:32.762529 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.763004 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.763746 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.763947 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.764024 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.764721 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:32.764722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:32.764722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:32.764727 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:32.764742 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:32.764746 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:32.764816 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.764878 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.764904 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 23:02:32.766636 9473 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 23:02:32.768181 9473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 23:02:32.775149 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44635 (local address 127.0.0.1:35860)
0810 23:02:32.770976 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.770986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.771021 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.771481 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.771482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.771483 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.771528 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.771529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772942 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.772943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.774089 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.774109 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.774908 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.774967 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.774971 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.775027 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.775083 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":53,"threads_started":1}
I20260810 23:02:32.775158 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.6.47.131:44635)
0810 23:02:32.770997 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.771002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.771033 (+ 31us) server_negotiation.cc:407] Connection header received
0810 23:02:32.771119 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.771122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.771155 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.771250 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.771611 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772834 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774182 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.774817 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774892 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.774984 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.775037 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.775073 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260810 23:02:32.775166 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.6.47.130:35187)
0810 23:02:32.770763 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.770768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.770955 (+ 187us) server_negotiation.cc:407] Connection header received
0810 23:02:32.771122 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.771125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.771161 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.771281 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.771687 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772804 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774141 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.774743 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774799 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.774885 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.775041 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.775080 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260810 23:02:32.775416 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.130:35187 (local address 127.0.0.1:37402)
0810 23:02:32.770842 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.770886 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.771002 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.771289 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.771292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.771295 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.771363 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.771365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772892 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.772893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.774054 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.774057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.774827 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.774907 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.774920 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.775264 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.775321 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":91,"threads_started":1}
I20260810 23:02:32.775959 10155 tablet_service.cc:1467] Processing CreateTablet for tablet 24e5559477454c6ea8c285a39853442f (DEFAULT_TABLE table=test-table [id=45882cf5715c4e468761fab90480a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.776098 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34159 (local address 127.0.0.1:45268)
0810 23:02:32.771336 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.771344 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.771404 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.771628 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.771629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.771632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.771687 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.771688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772686 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.772687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.774062 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.774064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.775886 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.775887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.775943 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.775947 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.776006 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.776043 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":55,"threads_started":1}
I20260810 23:02:32.776161 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.770845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.6.47.129:34159)
0810 23:02:32.771165 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.771169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.771437 (+ 268us) server_negotiation.cc:407] Connection header received
0810 23:02:32.771488 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.771490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.771506 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.771582 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.771748 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.772605 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.774936 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.775694 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.775760 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.775849 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.776034 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.776070 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":115,"threads_started":1}
I20260810 23:02:32.776367 10155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e5559477454c6ea8c285a39853442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.776855 9680 tablet_service.cc:1467] Processing CreateTablet for tablet 24e5559477454c6ea8c285a39853442f (DEFAULT_TABLE table=test-table [id=45882cf5715c4e468761fab90480a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.777189 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e5559477454c6ea8c285a39853442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.779336 10305 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Bootstrap starting.
I20260810 23:02:32.779902 9923 tablet_service.cc:1467] Processing CreateTablet for tablet 24e5559477454c6ea8c285a39853442f (DEFAULT_TABLE table=test-table [id=45882cf5715c4e468761fab90480a927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 23:02:32.780050 10305 tablet_bootstrap.cc:654] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.780041 10306 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: Bootstrap starting.
I20260810 23:02:32.780300 9923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24e5559477454c6ea8c285a39853442f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:32.780414 10305 log.cc:824] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.780828 10306 tablet_bootstrap.cc:654] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.781185 10306 log.cc:824] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.781318 10305 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: No bootstrap required, opened a new log
I20260810 23:02:32.781390 10305 ts_tablet_manager.cc:1397] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:02:32.781919 10306 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: No bootstrap required, opened a new log
I20260810 23:02:32.781987 10306 ts_tablet_manager.cc:1397] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 23:02:32.783898 10309 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Bootstrap starting.
I20260810 23:02:32.783948 10306 raft_consensus.cc:348] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.784098 10306 raft_consensus.cc:374] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.784184 10306 raft_consensus.cc:729] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b753b17294aa43e48bf6e4bebda7d202, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.784197 10305 raft_consensus.cc:348] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.784315 10306 consensus_queue.cc:260] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.784344 10305 raft_consensus.cc:374] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.784380 10305 raft_consensus.cc:729] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736bb3e799aa4932aab73a4d18a89557, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.784500 10305 consensus_queue.cc:260] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.784613 10306 ts_tablet_manager.cc:1428] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260810 23:02:32.784643 10309 tablet_bootstrap.cc:654] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:32.784798 10305 ts_tablet_manager.cc:1428] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 23:02:32.784827 10284 heartbeater.cc:503] Master 127.6.47.190:40579 was elected leader, sending a full tablet report...
I20260810 23:02:32.784981 10309 log.cc:824] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:32.785049 9801 heartbeater.cc:503] Master 127.6.47.190:40579 was elected leader, sending a full tablet report...
I20260810 23:02:32.785904 10309 tablet_bootstrap.cc:492] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: No bootstrap required, opened a new log
I20260810 23:02:32.785976 10309 ts_tablet_manager.cc:1397] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 23:02:32.787926 10309 raft_consensus.cc:348] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.788048 10309 raft_consensus.cc:374] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:32.788069 10309 raft_consensus.cc:729] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe503342529444809357008c7d87849b, State: Initialized, Role: FOLLOWER
I20260810 23:02:32.788179 10309 consensus_queue.cc:260] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.788443 10309 ts_tablet_manager.cc:1428] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 23:02:32.788519 10041 heartbeater.cc:503] Master 127.6.47.190:40579 was elected leader, sending a full tablet report...
W20260810 23:02:32.865173 10042 tablet.cc:2369] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 23:02:32.912678 10318 raft_consensus.cc:482] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 23:02:32.912802 10318 raft_consensus.cc:504] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.913249 10318 leader_election.cc:288] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b753b17294aa43e48bf6e4bebda7d202 (127.6.47.129:34159), 736bb3e799aa4932aab73a4d18a89557 (127.6.47.131:44635)
I20260810 23:02:32.917198 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.913507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:59765 (local address 127.6.47.129:34159)
0810 23:02:32.913551 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.913553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.913754 (+ 201us) server_negotiation.cc:407] Connection header received
0810 23:02:32.913794 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.913795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.913805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.913850 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.914014 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.914709 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916201 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.916796 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916891 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.916900 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.917062 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.917105 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:02:32.917198 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.913521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.47.130:44899 (local address 127.6.47.131:44635)
0810 23:02:32.913584 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.913586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.913598 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:02:32.913632 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.913635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.913650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.913702 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 23:02:32.913926 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.914685 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916200 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.916795 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916891 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.916900 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.917062 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.917130 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 23:02:32.917225 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.913426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.131:44635 (local address 127.6.47.130:44899)
0810 23:02:32.913485 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.913497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.913543 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.913737 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.913739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.913740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.913812 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.913816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.914793 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.914794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.916101 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.916104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.916899 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.916973 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.916980 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.917040 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.917095 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":44}
I20260810 23:02:32.917232 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.913466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34159 (local address 127.6.47.130:59765)
0810 23:02:32.913652 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.913660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:02:32.913737 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:02:32.913871 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:02:32.913872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:02:32.913874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 23:02:32.913946 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.913947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.914793 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.914794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.916101 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:02:32.916104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.916911 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.916911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:02:32.916973 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.916980 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 23:02:32.917040 (+ 60us) client_negotiation.cc:240] Negotiation successful
0810 23:02:32.917095 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":82,"thread_start_us":124,"threads_started":1}
I20260810 23:02:32.917593 10194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e5559477454c6ea8c285a39853442f" candidate_uuid: "fe503342529444809357008c7d87849b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736bb3e799aa4932aab73a4d18a89557" is_pre_election: true
I20260810 23:02:32.917702 9722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e5559477454c6ea8c285a39853442f" candidate_uuid: "fe503342529444809357008c7d87849b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b753b17294aa43e48bf6e4bebda7d202" is_pre_election: true
I20260810 23:02:32.917773 10194 raft_consensus.cc:2454] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe503342529444809357008c7d87849b in term 0.
I20260810 23:02:32.917888 9722 raft_consensus.cc:2454] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe503342529444809357008c7d87849b in term 0.
I20260810 23:02:32.918023 9828 leader_election.cc:302] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736bb3e799aa4932aab73a4d18a89557, fe503342529444809357008c7d87849b; no voters:
I20260810 23:02:32.918164 10318 raft_consensus.cc:2781] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 23:02:32.918226 10318 raft_consensus.cc:482] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 23:02:32.918251 10318 raft_consensus.cc:3037] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.918975 10318 raft_consensus.cc:504] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.919131 10318 leader_election.cc:288] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [CANDIDATE]: Term 1 election: Requested vote from peers b753b17294aa43e48bf6e4bebda7d202 (127.6.47.129:34159), 736bb3e799aa4932aab73a4d18a89557 (127.6.47.131:44635)
I20260810 23:02:32.919309 10194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e5559477454c6ea8c285a39853442f" candidate_uuid: "fe503342529444809357008c7d87849b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736bb3e799aa4932aab73a4d18a89557"
I20260810 23:02:32.919308 9722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24e5559477454c6ea8c285a39853442f" candidate_uuid: "fe503342529444809357008c7d87849b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b753b17294aa43e48bf6e4bebda7d202"
I20260810 23:02:32.919390 10194 raft_consensus.cc:3037] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.919390 9722 raft_consensus.cc:3037] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:32.920313 9722 raft_consensus.cc:2454] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe503342529444809357008c7d87849b in term 1.
I20260810 23:02:32.920313 10194 raft_consensus.cc:2454] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe503342529444809357008c7d87849b in term 1.
I20260810 23:02:32.920553 9831 leader_election.cc:302] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b753b17294aa43e48bf6e4bebda7d202, fe503342529444809357008c7d87849b; no voters:
I20260810 23:02:32.920662 10318 raft_consensus.cc:2781] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:32.920853 10318 raft_consensus.cc:686] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 1 LEADER]: Becoming Leader. State: Replica: fe503342529444809357008c7d87849b, State: Running, Role: LEADER
I20260810 23:02:32.920962 10318 consensus_queue.cc:237] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } }
I20260810 23:02:32.921849 9475 catalog_manager.cc:5495] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b reported cstate change: term changed from 0 to 1, leader changed from <none> to fe503342529444809357008c7d87849b (127.6.47.130). New cstate: current_term: 1 leader_uuid: "fe503342529444809357008c7d87849b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe503342529444809357008c7d87849b" member_type: VOTER last_known_addr { host: "127.6.47.130" port: 35187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 } health_report { overall_health: UNKNOWN } } }
I20260810 23:02:32.955219 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.952742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.6.47.130:35187)
0810 23:02:32.952810 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:32.952812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:32.952822 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:02:32.952893 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:32.952893 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:32.952913 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:32.952979 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:02:32.953104 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.953557 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.954378 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:32.954584 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:32.954685 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:32.954857 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:02:32.954969 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:02:32.955070 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:32.955088 (+ 18us) server_negotiation.cc:299] Negotiation successful
0810 23:02:32.955110 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:02:32.957201 10194 raft_consensus.cc:1261] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 1 FOLLOWER]: Refusing update from remote peer fe503342529444809357008c7d87849b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:32.957201 9722 raft_consensus.cc:1261] T 24e5559477454c6ea8c285a39853442f P b753b17294aa43e48bf6e4bebda7d202 [term 1 FOLLOWER]: Refusing update from remote peer fe503342529444809357008c7d87849b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:02:32.957643 10320 consensus_queue.cc:1035] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b753b17294aa43e48bf6e4bebda7d202" member_type: VOTER last_known_addr { host: "127.6.47.129" port: 34159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:32.957841 10318 consensus_queue.cc:1035] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [LEADER]: Connected to new peer: Peer: permanent_uuid: "736bb3e799aa4932aab73a4d18a89557" member_type: VOTER last_known_addr { host: "127.6.47.131" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:02:32.960919 10327 mvcc.cc:205] Tried to move back new op lower bound from 7317106495309594624 to 7317106495165009920. Current Snapshot: MvccSnapshot[applied={T|T < 7317106495309594624}]
I20260810 23:02:32.963002 9473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:32.963117 9473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 23:02:32.963907 9473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41003e5b4f0146299025840428fbb525: Sending DeleteTablet for 3 replicas of tablet 24e5559477454c6ea8c285a39853442f
I20260810 23:02:32.964278 9923 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e5559477454c6ea8c285a39853442f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:32 UTC) from {username='slave'} at 127.0.0.1:37402
I20260810 23:02:32.964323 10155 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e5559477454c6ea8c285a39853442f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:32 UTC) from {username='slave'} at 127.0.0.1:35860
I20260810 23:02:32.964346 9680 tablet_service.cc:1514] Processing DeleteTablet for tablet 24e5559477454c6ea8c285a39853442f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 23:02:32 UTC) from {username='slave'} at 127.0.0.1:45268
I20260810 23:02:32.964553 10337 tablet_replica.cc:330] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: stopping tablet replica
I20260810 23:02:32.964578 10341 tablet_replica.cc:330] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: stopping tablet replica
I20260810 23:02:32.964664 10337 raft_consensus.cc:2229] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 1 LEADER]: Raft consensus shutting down.
I20260810 23:02:32.964787 10337 raft_consensus.cc:2258] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:32.964799 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9574
I20260810 23:02:32.964799 10341 raft_consensus.cc:2229] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 23:02:32.965013 10341 raft_consensus.cc:2258] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 23:02:32.968046 10341 ts_tablet_manager.cc:1905] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:32.970283 10341 ts_tablet_manager.cc:1918] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:32.970415 10341 log.cc:1196] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-2/wal/wals/24e5559477454c6ea8c285a39853442f
I20260810 23:02:32.970745 10341 ts_tablet_manager.cc:1939] T 24e5559477454c6ea8c285a39853442f P 736bb3e799aa4932aab73a4d18a89557: Deleting consensus metadata
I20260810 23:02:32.971493 9444 catalog_manager.cc:4841] TS 736bb3e799aa4932aab73a4d18a89557 (127.6.47.131:44635): tablet 24e5559477454c6ea8c285a39853442f (table test-table [id=45882cf5715c4e468761fab90480a927]) successfully deleted
I20260810 23:02:32.974282 10337 ts_tablet_manager.cc:1905] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 23:02:32.975939 10337 ts_tablet_manager.cc:1918] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 23:02:32.976020 10337 log.cc:1196] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Deleting WAL directory at /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786402701710467-6334-0/minicluster-data/ts-1/wal/wals/24e5559477454c6ea8c285a39853442f
I20260810 23:02:32.976275 10337 ts_tablet_manager.cc:1939] T 24e5559477454c6ea8c285a39853442f P fe503342529444809357008c7d87849b: Deleting consensus metadata
I20260810 23:02:32.976773 9446 catalog_manager.cc:4841] TS fe503342529444809357008c7d87849b (127.6.47.130:35187): tablet 24e5559477454c6ea8c285a39853442f (table test-table [id=45882cf5715c4e468761fab90480a927]) successfully deleted
W20260810 23:02:32.979990 9446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.47.129:34159 (error 108)
I20260810 23:02:32.980590 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:32.980383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.129:34159 (local address 127.0.0.1:45278)
0810 23:02:32.980448 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0810 23:02:32.980532 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.129:34159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:02:32.980623 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9805
W20260810 23:02:32.980832 9446 catalog_manager.cc:4584] TS b753b17294aa43e48bf6e4bebda7d202 (127.6.47.129:34159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24e5559477454c6ea8c285a39853442f: Network error: Client connection negotiation failed: client connection to 127.6.47.129:34159: connect: Connection refused (error 111)
I20260810 23:02:32.992940 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10044
I20260810 23:02:33.005030 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 9419
2026-08-10T23:02:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (892 ms)
[----------] 6 tests from EncryptionPolicyTest (9533 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:33 dist-test-slave-rwrg krb5kdc[10350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:33 dist-test-slave-rwrg krb5kdc[10350](info): set up 2 sockets
Aug 10 23:02:33 dist-test-slave-rwrg krb5kdc[10350](info): commencing operation
krb5kdc: starting...
W20260810 23:02:34.327999 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:34.451761 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:34.533689 10379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:34.533949 10379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:34.533990 10379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:34.535254 10379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:34.535326 10379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:34.535351 10379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:34.535380 10379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:34.535411 10379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:34.536924 10379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:34.537216 10379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:34.537452 10379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:34.542256 10394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:34.542387 10379 server_base.cc:1034] running on GCE node
W20260810 23:02:34.542224 10385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:34.542232 10392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:34.542719 10379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:34.543004 10379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:34.544143 10379 hybrid_clock.cc:648] HybridClock initialized: now 1786402954544125 us; error 36 us; skew 500 ppm
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402954, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:34.551079 10379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:34.551581 10379 webserver.cc:460] Webserver started at http://127.0.0.1:43579/ using document root <none> and password file <none>
I20260810 23:02:34.551832 10379 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:34.551905 10379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:34.552058 10379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:34.553186 10379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "4c7f8a7e9643406a92dde6460e541812"
format_stamp: "Formatted at 2026-08-10 23:02:34 on dist-test-slave-rwrg"
server_key: "5ed51c5397486e09c29c6fbd7fff52a8"
server_key_iv: "d89ead3d7e1b61beefe3dee5a429c991"
server_key_version: "encryptionkey@0"
I20260810 23:02:34.553553 10379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "4c7f8a7e9643406a92dde6460e541812"
format_stamp: "Formatted at 2026-08-10 23:02:34 on dist-test-slave-rwrg"
server_key: "5ed51c5397486e09c29c6fbd7fff52a8"
server_key_iv: "d89ead3d7e1b61beefe3dee5a429c991"
server_key_version: "encryptionkey@0"
I20260810 23:02:34.554895 10379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:02:34.555655 10405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:34.555830 10379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:34.555938 10379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "4c7f8a7e9643406a92dde6460e541812"
format_stamp: "Formatted at 2026-08-10 23:02:34 on dist-test-slave-rwrg"
server_key: "5ed51c5397486e09c29c6fbd7fff52a8"
server_key_iv: "d89ead3d7e1b61beefe3dee5a429c991"
server_key_version: "encryptionkey@0"
I20260810 23:02:34.556030 10379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:34.575050 10379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:34.577229 10379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:34.577396 10379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:34.581717 10379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42099
I20260810 23:02:34.581768 10507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42099 every 8 connection(s)
I20260810 23:02:34.582149 10379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:34.583184 10509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:34.585494 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812: Bootstrap starting.
I20260810 23:02:34.586042 10509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:34.586295 10509 log.cc:824] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:34.586809 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10379
I20260810 23:02:34.586895 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:34.587067 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812: No bootstrap required, opened a new log
I20260810 23:02:34.587090 6334 external_mini_cluster.cc:1428] Setting key 74ff3679bd624423e8b6459755d57882
I20260810 23:02:34.589521 10509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260810 23:02:34.589718 10509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:34.589763 10509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7f8a7e9643406a92dde6460e541812, State: Initialized, Role: FOLLOWER
I20260810 23:02:34.589924 10509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260810 23:02:34.590003 10509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:34.590035 10509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:34.590081 10509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:34.590807 10509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260810 23:02:34.590920 10509 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c7f8a7e9643406a92dde6460e541812; no voters:
I20260810 23:02:34.591071 10509 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:34.591135 10520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:34.591331 10509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:34.591357 10520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7f8a7e9643406a92dde6460e541812, State: Running, Role: LEADER
I20260810 23:02:34.591532 10520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } }
I20260810 23:02:34.591960 10523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7f8a7e9643406a92dde6460e541812. Latest consensus state: current_term: 1 leader_uuid: "4c7f8a7e9643406a92dde6460e541812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } } }
I20260810 23:02:34.592056 10523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:34.591948 10521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c7f8a7e9643406a92dde6460e541812" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7f8a7e9643406a92dde6460e541812" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42099 } } }
I20260810 23:02:34.592155 10521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:34.592628 10538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:34.593226 10538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:34.594918 10538 catalog_manager.cc:1347] Generated new cluster ID: 32d1f374c61f49c1a78dcc3ee152c9c5
I20260810 23:02:34.594974 10538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:02:34.599507 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:34.587710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41685 (local address 127.0.0.1:42099)
0810 23:02:34.587940 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:34.587950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:34.587984 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:02:34.588682 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:34.588693 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:34.588760 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:34.588916 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:34.589534 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:34.591215 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:34.592403 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:34.593175 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:34.593306 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:34.595982 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:34.595986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:34.595987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:34.595998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:34.597783 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:34.598196 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:34.598199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:34.598199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:34.598293 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:34.598539 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:34.598540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:34.598540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:34.598782 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:34.598957 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:34.599092 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 23:02:34.599210 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260810 23:02:34.600585 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:34.603204 10538 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:34.603737 10538 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:34.606462 10538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7f8a7e9643406a92dde6460e541812: Generated new TSK 0
I20260810 23:02:34.606645 10538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:34.611956 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:34.604860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42077 (local address 127.0.0.1:42099)
0810 23:02:34.604896 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:34.604898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:34.605057 (+ 159us) server_negotiation.cc:407] Connection header received
0810 23:02:34.605121 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:34.605122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:34.605131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:34.605192 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:34.605853 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:34.606566 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:34.607611 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:34.607811 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:34.607868 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:34.609289 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:34.609291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:34.609291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:34.609298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:34.610711 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:34.611135 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:34.611135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:34.611136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:34.611189 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:34.611420 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:34.611421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:34.611421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:34.611513 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:34.611587 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:34.611762 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 23:02:34.611813 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:02:34.614089 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10379
W20260810 23:02:34.632252 10343 connection.cc:535] client connection to 127.0.0.1:42099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42099): Connection reset by peer (error 104)
2026-08-10T23:02:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1615 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10563](info): set up 2 sockets
Aug 10 23:02:34 dist-test-slave-rwrg krb5kdc[10563](info): commencing operation
krb5kdc: starting...
W20260810 23:02:35.921856 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:36 dist-test-slave-rwrg krb5kdc[10563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:36.051923 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:36.134331 10583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:36.134574 10583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:36.134624 10583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:36.135787 10583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:36.135856 10583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:36.135881 10583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:36.135910 10583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:36.135941 10583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:36.137468 10583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:36.137777 10583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:36.138025 10583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:36.142968 10594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:36.142987 10596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:36.143056 10591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:36.143288 10583 server_base.cc:1034] running on GCE node
I20260810 23:02:36.143501 10583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:36.143738 10583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:36.144873 10583 hybrid_clock.cc:648] HybridClock initialized: now 1786402956144855 us; error 33 us; skew 500 ppm
Aug 10 23:02:36 dist-test-slave-rwrg krb5kdc[10563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402956, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:36.147326 10583 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:36.147701 10583 webserver.cc:460] Webserver started at http://127.0.0.1:36823/ using document root <none> and password file <none>
I20260810 23:02:36.147904 10583 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:36.147944 10583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:36.148033 10583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:36.149122 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "4f8c6dea6a1a4e55a8091baf9981d626"
format_stamp: "Formatted at 2026-08-10 23:02:36 on dist-test-slave-rwrg"
server_key: "82879e8f9cd5c933e2bfb4967c84a50d"
server_key_iv: "b6265ec102fa4b7f989b259f7aab0eb3"
server_key_version: "encryptionkey@0"
I20260810 23:02:36.149459 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "4f8c6dea6a1a4e55a8091baf9981d626"
format_stamp: "Formatted at 2026-08-10 23:02:36 on dist-test-slave-rwrg"
server_key: "82879e8f9cd5c933e2bfb4967c84a50d"
server_key_iv: "b6265ec102fa4b7f989b259f7aab0eb3"
server_key_version: "encryptionkey@0"
I20260810 23:02:36.150743 10583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:36.151458 10603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:36.151628 10583 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:36.151739 10583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "4f8c6dea6a1a4e55a8091baf9981d626"
format_stamp: "Formatted at 2026-08-10 23:02:36 on dist-test-slave-rwrg"
server_key: "82879e8f9cd5c933e2bfb4967c84a50d"
server_key_iv: "b6265ec102fa4b7f989b259f7aab0eb3"
server_key_version: "encryptionkey@0"
I20260810 23:02:36.151814 10583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:36.174118 10583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:36.176663 10583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:36.176858 10583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:36.181002 10583 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41917
I20260810 23:02:36.181041 10696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41917 every 8 connection(s)
I20260810 23:02:36.181406 10583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:36.182389 10698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:36.184765 10698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626: Bootstrap starting.
I20260810 23:02:36.185379 10698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:36.185725 10698 log.cc:824] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:36.186453 10698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626: No bootstrap required, opened a new log
I20260810 23:02:36.187691 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10583
I20260810 23:02:36.187822 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:36.187991 6334 external_mini_cluster.cc:1428] Setting key a8adb4a5b6ffe319c8959ebc56ae8f27
I20260810 23:02:36.188297 10698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } }
I20260810 23:02:36.188520 10698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:36.188573 10698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8c6dea6a1a4e55a8091baf9981d626, State: Initialized, Role: FOLLOWER
I20260810 23:02:36.188714 10698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } }
I20260810 23:02:36.188818 10698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:36.188854 10698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:36.188908 10698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:36.189975 10698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } }
I20260810 23:02:36.190104 10698 leader_election.cc:302] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f8c6dea6a1a4e55a8091baf9981d626; no voters:
I20260810 23:02:36.190299 10698 leader_election.cc:288] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:36.190346 10705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:36.190521 10705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8c6dea6a1a4e55a8091baf9981d626, State: Running, Role: LEADER
I20260810 23:02:36.190613 10698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:36.190634 10705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } }
I20260810 23:02:36.190987 10705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f8c6dea6a1a4e55a8091baf9981d626. Latest consensus state: current_term: 1 leader_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } } }
I20260810 23:02:36.191071 10705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:36.191002 10706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8c6dea6a1a4e55a8091baf9981d626" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41917 } } }
I20260810 23:02:36.191254 10706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:36.191285 10713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:36.191874 10713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:36.193682 10713 catalog_manager.cc:1347] Generated new cluster ID: c21c25a3eedd4feaa9057037f2a60ff6
I20260810 23:02:36.193730 10713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:36.193951 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:36.188569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41463 (local address 127.0.0.1:41917)
0810 23:02:36.188756 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:36.188764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:36.188790 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:02:36.188912 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:36.188919 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:36.189005 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:36.189126 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:36.189420 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:36.190461 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:36.191588 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:36.192352 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:36.192458 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:36.193326 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:36.193330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:36.193330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:36.193511 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:36.193539 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:36.193550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:36.193659 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:36.193748 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 23:02:36.193777 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260810 23:02:36.197474 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:36.201808 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:36.198935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58847 (local address 127.0.0.1:41917)
0810 23:02:36.198985 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:36.198987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:36.199096 (+ 109us) server_negotiation.cc:407] Connection header received
0810 23:02:36.199178 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:36.199179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:36.199189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:36.199246 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:36.199398 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:36.199855 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:36.200584 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:36.200813 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:36.200892 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:36.201552 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:36.201553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:36.201553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:36.201558 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:36.201570 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:36.201574 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:36.201626 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:36.201704 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 23:02:36.201717 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 23:02:36.240263 10713 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:36.240899 10713 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:36.255686 10713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f8c6dea6a1a4e55a8091baf9981d626: Generated new TSK 0
I20260810 23:02:36.255906 10713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:36.268014 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10583
W20260810 23:02:36.282096 10559 connection.cc:535] client connection to 127.0.0.1:41917 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41917): Connection reset by peer (error 104)
2026-08-10T23:02:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:36 dist-test-slave-rwrg krb5kdc[10750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:36 dist-test-slave-rwrg krb5kdc[10750](info): set up 2 sockets
Aug 10 23:02:36 dist-test-slave-rwrg krb5kdc[10750](info): commencing operation
krb5kdc: starting...
W20260810 23:02:37.620743 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:37 dist-test-slave-rwrg krb5kdc[10750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:37.747361 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:37.829561 10779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:37.829839 10779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:37.829882 10779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:37.831058 10779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:37.831151 10779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:37.831176 10779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:37.831205 10779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:37.831233 10779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:37.832788 10779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:37.833086 10779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:37.833329 10779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:37.838212 10789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:37.838496 10779 server_base.cc:1034] running on GCE node
W20260810 23:02:37.838212 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:37.838277 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:37.838937 10779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:37.839270 10779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:37.840456 10779 hybrid_clock.cc:648] HybridClock initialized: now 1786402957840454 us; error 42 us; skew 500 ppm
Aug 10 23:02:37 dist-test-slave-rwrg krb5kdc[10750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:37.843163 10779 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:37.843669 10779 webserver.cc:460] Webserver started at http://127.0.0.1:42081/ using document root <none> and password file <none>
I20260810 23:02:37.843911 10779 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:37.843981 10779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:37.844120 10779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:37.845371 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "4fa7c6159db142b0aa2a9846b4c6da56"
format_stamp: "Formatted at 2026-08-10 23:02:37 on dist-test-slave-rwrg"
server_key: "acd9d6d0439af5df73ca229361485dd6"
server_key_iv: "b0fcefb042c4db8e49233fb47be729af"
server_key_version: "encryptionkey@0"
I20260810 23:02:37.845760 10779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "4fa7c6159db142b0aa2a9846b4c6da56"
format_stamp: "Formatted at 2026-08-10 23:02:37 on dist-test-slave-rwrg"
server_key: "acd9d6d0439af5df73ca229361485dd6"
server_key_iv: "b0fcefb042c4db8e49233fb47be729af"
server_key_version: "encryptionkey@0"
I20260810 23:02:37.847148 10779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:02:37.847955 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:37.848205 10779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:37.848320 10779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "4fa7c6159db142b0aa2a9846b4c6da56"
format_stamp: "Formatted at 2026-08-10 23:02:37 on dist-test-slave-rwrg"
server_key: "acd9d6d0439af5df73ca229361485dd6"
server_key_iv: "b0fcefb042c4db8e49233fb47be729af"
server_key_version: "encryptionkey@0"
I20260810 23:02:37.848397 10779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:37.851719 10779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:37.853698 10779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:37.853854 10779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:37.858129 10779 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40343
I20260810 23:02:37.858156 10905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40343 every 8 connection(s)
I20260810 23:02:37.858482 10779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:37.859421 10906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:37.861706 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56: Bootstrap starting.
I20260810 23:02:37.862406 10906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:37.862596 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10779
I20260810 23:02:37.862721 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:37.862746 10906 log.cc:824] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:37.862903 6334 external_mini_cluster.cc:1428] Setting key 86f3fcfa69b0dff559e008b94b6277fc
I20260810 23:02:37.863621 10906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56: No bootstrap required, opened a new log
I20260810 23:02:37.865489 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:37.863381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46519 (local address 127.0.0.1:40343)
0810 23:02:37.863574 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:37.863583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:37.863617 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:02:37.864090 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:37.864097 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:37.864121 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:37.864279 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:37.865051 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:37.865055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:37.865055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:37.865059 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:37.865121 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:37.865131 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:37.865241 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:37.865288 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 23:02:37.865328 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260810 23:02:37.866514 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:37.866394 10906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } }
I20260810 23:02:37.866573 10906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:37.866626 10906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fa7c6159db142b0aa2a9846b4c6da56, State: Initialized, Role: FOLLOWER
I20260810 23:02:37.866750 10906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } }
I20260810 23:02:37.866843 10906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:37.866888 10906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:37.866936 10906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:37.867488 10906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } }
I20260810 23:02:37.867620 10906 leader_election.cc:302] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fa7c6159db142b0aa2a9846b4c6da56; no voters:
I20260810 23:02:37.867786 10906 leader_election.cc:288] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:37.867842 10921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:37.868014 10921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [term 1 LEADER]: Becoming Leader. State: Replica: 4fa7c6159db142b0aa2a9846b4c6da56, State: Running, Role: LEADER
I20260810 23:02:37.868058 10906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:37.868124 10921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } }
I20260810 23:02:37.868582 10921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } } }
I20260810 23:02:37.868667 10921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:37.868842 10921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fa7c6159db142b0aa2a9846b4c6da56. Latest consensus state: current_term: 1 leader_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fa7c6159db142b0aa2a9846b4c6da56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40343 } } }
I20260810 23:02:37.868960 10921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:37.869149 10942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:37.869352 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:37.867993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41957 (local address 127.0.0.1:40343)
0810 23:02:37.868036 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:37.868038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:37.868210 (+ 172us) server_negotiation.cc:407] Connection header received
0810 23:02:37.868292 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:37.868293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:37.868298 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:37.868349 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:37.868995 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:37.868996 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:37.868996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:37.869001 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:37.869015 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:37.869019 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:37.869067 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:37.869286 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 23:02:37.869300 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 23:02:37.869884 10942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:37.871623 10942 catalog_manager.cc:1347] Generated new cluster ID: 8b51c4281da6450cae561aca1cc353e0
I20260810 23:02:37.871683 10942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:37.894527 10942 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:37.894713 10942 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:37.911722 10942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fa7c6159db142b0aa2a9846b4c6da56: Generated new TSK 0
I20260810 23:02:37.911937 10942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:37.920655 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10779
2026-08-10T23:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:37 dist-test-slave-rwrg krb5kdc[10969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:37 dist-test-slave-rwrg krb5kdc[10969](info): set up 2 sockets
Aug 10 23:02:37 dist-test-slave-rwrg krb5kdc[10969](info): commencing operation
krb5kdc: starting...
W20260810 23:02:39.239899 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[10969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:39.365460 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:39.446651 10991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:39.446856 10991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:39.446877 10991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:39.448211 10991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:39.448258 10991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:39.448273 10991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:39.448285 10991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:39.448314 10991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:39.449834 10991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:39.450093 10991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:39.450321 10991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:39.455256 11001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:39.455312 11003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:39.455291 11005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:39.455327 10991 server_base.cc:1034] running on GCE node
I20260810 23:02:39.455684 10991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:39.455989 10991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:39.457156 10991 hybrid_clock.cc:648] HybridClock initialized: now 1786402959457140 us; error 44 us; skew 500 ppm
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[10969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:39.459622 10991 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:39.460088 10991 webserver.cc:460] Webserver started at http://127.0.0.1:34455/ using document root <none> and password file <none>
I20260810 23:02:39.460282 10991 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:39.460322 10991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:39.460409 10991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:39.461462 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "d6be3acd96514faabb0c877b8808fa16"
format_stamp: "Formatted at 2026-08-10 23:02:39 on dist-test-slave-rwrg"
server_key: "6b1ee30908ba254111569afb2a4fb70f"
server_key_iv: "4be1b478b2569cdf06bb39f1ceb96d71"
server_key_version: "encryptionkey@0"
I20260810 23:02:39.461822 10991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "d6be3acd96514faabb0c877b8808fa16"
format_stamp: "Formatted at 2026-08-10 23:02:39 on dist-test-slave-rwrg"
server_key: "6b1ee30908ba254111569afb2a4fb70f"
server_key_iv: "4be1b478b2569cdf06bb39f1ceb96d71"
server_key_version: "encryptionkey@0"
I20260810 23:02:39.463186 10991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:39.463955 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:39.464124 10991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:39.464223 10991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "d6be3acd96514faabb0c877b8808fa16"
format_stamp: "Formatted at 2026-08-10 23:02:39 on dist-test-slave-rwrg"
server_key: "6b1ee30908ba254111569afb2a4fb70f"
server_key_iv: "4be1b478b2569cdf06bb39f1ceb96d71"
server_key_version: "encryptionkey@0"
I20260810 23:02:39.464278 10991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:39.471010 10991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:39.473272 10991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:39.473382 10991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:39.477349 10991 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37967
I20260810 23:02:39.477420 11110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37967 every 8 connection(s)
I20260810 23:02:39.477732 10991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:39.478729 11111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:39.480297 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 10991
I20260810 23:02:39.480420 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:39.480674 6334 external_mini_cluster.cc:1428] Setting key 4134c92322900f6b3b7cb0d100659d25
I20260810 23:02:39.481719 11111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: Bootstrap starting.
I20260810 23:02:39.482425 11111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:39.482733 11111 log.cc:824] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:39.483494 11111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: No bootstrap required, opened a new log
I20260810 23:02:39.485983 11111 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260810 23:02:39.486143 11111 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:39.486167 11111 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6be3acd96514faabb0c877b8808fa16, State: Initialized, Role: FOLLOWER
I20260810 23:02:39.486315 11111 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260810 23:02:39.486413 11111 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:39.486474 11111 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:39.486524 11111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:39.487058 11111 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260810 23:02:39.487178 11111 leader_election.cc:302] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6be3acd96514faabb0c877b8808fa16; no voters:
I20260810 23:02:39.487372 11111 leader_election.cc:288] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:39.487401 11118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:39.487516 11118 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [term 1 LEADER]: Becoming Leader. State: Replica: d6be3acd96514faabb0c877b8808fa16, State: Running, Role: LEADER
I20260810 23:02:39.487597 11118 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } }
I20260810 23:02:39.487644 11111 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:39.487932 11118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6be3acd96514faabb0c877b8808fa16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } } }
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[10969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:02:39.488023 11118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:39.488165 11118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6be3acd96514faabb0c877b8808fa16. Latest consensus state: current_term: 1 leader_uuid: "d6be3acd96514faabb0c877b8808fa16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6be3acd96514faabb0c877b8808fa16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37967 } } }
I20260810 23:02:39.488238 11118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16 [sys.catalog]: This master's current role is: LEADER
W20260810 23:02:39.489823 11141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:02:39.489885 11141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:02:39.490509 11125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:39.490713 11125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:39.492673 11125 catalog_manager.cc:1347] Generated new cluster ID: cbfa4e9f77924bc7bdcb3f5b339633bd
I20260810 23:02:39.492730 11125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:39.493355 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:39.481227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40573 (local address 127.0.0.1:37967)
0810 23:02:39.481445 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:39.481455 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:39.481484 (+ 29us) server_negotiation.cc:407] Connection header received
0810 23:02:39.481995 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:39.482007 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:39.482084 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:39.482245 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:39.482950 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:39.484254 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:39.485103 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:39.485850 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:39.485957 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:39.488617 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:39.488622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:39.488622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:39.488634 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:39.491231 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:39.491656 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:39.491658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:39.491658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:39.491745 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:39.491969 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:39.491971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:39.491971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:39.492283 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:39.492962 (+ 679us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:39.493030 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 23:02:39.493153 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":60,"threads_started":1}
I20260810 23:02:39.494073 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:39.503679 11116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:39.495495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57423 (local address 127.0.0.1:37967)
0810 23:02:39.495524 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:39.495526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:39.496788 (+ 1262us) server_negotiation.cc:407] Connection header received
0810 23:02:39.496817 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:39.496818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:39.496828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:39.496900 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:39.497478 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:39.498237 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:39.499202 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:39.499508 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:39.499581 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:39.500922 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:39.500923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:39.500923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:39.500930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:39.502555 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:39.502922 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:39.502923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:39.502924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:39.502985 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:39.503177 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:39.503178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:39.503178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:39.503306 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:39.503403 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:39.503501 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 23:02:39.503579 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260810 23:02:39.515411 11125 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:39.516136 11125 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:39.522363 11125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6be3acd96514faabb0c877b8808fa16: Generated new TSK 0
I20260810 23:02:39.522552 11125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:39.526015 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 10991
2026-08-10T23:02:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1617 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[11163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[11163](info): set up 2 sockets
Aug 10 23:02:39 dist-test-slave-rwrg krb5kdc[11163](info): commencing operation
krb5kdc: starting...
W20260810 23:02:40.827435 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:40 dist-test-slave-rwrg krb5kdc[11163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:40.948676 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:41.030759 11187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:41.030966 11187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:41.030988 11187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:41.032025 11187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:41.032059 11187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:41.032073 11187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:41.032084 11187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:41.032096 11187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:41.033542 11187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:41.033751 11187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:41.033918 11187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:41.038895 11198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:41.038938 11197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:41.039073 11187 server_base.cc:1034] running on GCE node
W20260810 23:02:41.038950 11201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:41.039444 11187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:41.039795 11187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:41.040963 11187 hybrid_clock.cc:648] HybridClock initialized: now 1786402961040943 us; error 37 us; skew 500 ppm
Aug 10 23:02:41 dist-test-slave-rwrg krb5kdc[11163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402961, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:41.043449 11187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:41.043956 11187 webserver.cc:460] Webserver started at http://127.0.0.1:36041/ using document root <none> and password file <none>
I20260810 23:02:41.044196 11187 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:41.044267 11187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:41.044425 11187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:41.045528 11187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "bd268d551c744d40bcb27b6edecac75b"
format_stamp: "Formatted at 2026-08-10 23:02:41 on dist-test-slave-rwrg"
server_key: "6b0160dd46aa15f3ec64ebe58a8b829d"
server_key_iv: "130698f5d067ebabb735a24d68cc690f"
server_key_version: "encryptionkey@0"
I20260810 23:02:41.045938 11187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "bd268d551c744d40bcb27b6edecac75b"
format_stamp: "Formatted at 2026-08-10 23:02:41 on dist-test-slave-rwrg"
server_key: "6b0160dd46aa15f3ec64ebe58a8b829d"
server_key_iv: "130698f5d067ebabb735a24d68cc690f"
server_key_version: "encryptionkey@0"
I20260810 23:02:41.047290 11187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:41.048046 11214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:41.048237 11187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:41.048353 11187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "bd268d551c744d40bcb27b6edecac75b"
format_stamp: "Formatted at 2026-08-10 23:02:41 on dist-test-slave-rwrg"
server_key: "6b0160dd46aa15f3ec64ebe58a8b829d"
server_key_iv: "130698f5d067ebabb735a24d68cc690f"
server_key_version: "encryptionkey@0"
I20260810 23:02:41.048431 11187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:41.092643 11187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:41.094681 11187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:41.094857 11187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:41.099222 11187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37873
I20260810 23:02:41.099261 11297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37873 every 8 connection(s)
I20260810 23:02:41.099649 11187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:41.100767 11302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:41.103118 11302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: Bootstrap starting.
I20260810 23:02:41.103682 11302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:41.103922 11302 log.cc:824] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:41.104532 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11187
I20260810 23:02:41.104655 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:41.104765 11302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: No bootstrap required, opened a new log
I20260810 23:02:41.104856 6334 external_mini_cluster.cc:1428] Setting key 412b4af76c803fd9c64ec1cfa0a1a8b7
I20260810 23:02:41.107447 11302 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260810 23:02:41.107673 11302 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:41.107720 11302 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd268d551c744d40bcb27b6edecac75b, State: Initialized, Role: FOLLOWER
I20260810 23:02:41.107844 11302 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260810 23:02:41.107923 11302 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:41.107952 11302 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:41.107996 11302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:41.108719 11302 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260810 23:02:41.108817 11302 leader_election.cc:302] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd268d551c744d40bcb27b6edecac75b; no voters:
I20260810 23:02:41.109050 11302 leader_election.cc:288] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:41.109073 11309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:41.109202 11309 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [term 1 LEADER]: Becoming Leader. State: Replica: bd268d551c744d40bcb27b6edecac75b, State: Running, Role: LEADER
I20260810 23:02:41.109350 11302 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:41.109364 11309 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } }
I20260810 23:02:41.110119 11311 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd268d551c744d40bcb27b6edecac75b. Latest consensus state: current_term: 1 leader_uuid: "bd268d551c744d40bcb27b6edecac75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260810 23:02:41.110229 11311 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:41.110576 11310 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd268d551c744d40bcb27b6edecac75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd268d551c744d40bcb27b6edecac75b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37873 } } }
I20260810 23:02:41.110651 11310 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:41.110888 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:41.105396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60777 (local address 127.0.0.1:37873)
0810 23:02:41.105599 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:41.105609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:41.105646 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:02:41.106124 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:41.106131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:41.106177 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:41.106291 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:41.106540 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:41.107430 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:41.108586 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:41.109079 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:41.109541 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:41.109960 (+ 419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:41.109964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:41.109964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:41.110187 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:41.110227 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:41.110243 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:41.110555 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:41.110590 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 23:02:41.110661 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":113,"threads_started":1}
W20260810 23:02:41.111855 11336 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:02:41.111984 11336 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:02:41.112061 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:41.112094 11325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:41.112253 11325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:41.114148 11325 catalog_manager.cc:1347] Generated new cluster ID: db042ad8ae3343a89ba504a00e050c53
I20260810 23:02:41.114204 11325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:41.116422 11308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:41.113531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38091 (local address 127.0.0.1:37873)
0810 23:02:41.113562 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:41.113563 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:41.113684 (+ 121us) server_negotiation.cc:407] Connection header received
0810 23:02:41.113752 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:41.113753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:41.113761 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:41.113833 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:41.113978 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:41.114492 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:41.115208 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:41.115406 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:41.115484 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:41.116141 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:41.116142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:41.116143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:41.116147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:41.116160 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:41.116163 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:41.116252 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:41.116327 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 23:02:41.116346 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260810 23:02:41.141853 11325 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:41.142592 11325 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:41.146533 11325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd268d551c744d40bcb27b6edecac75b: Generated new TSK 0
I20260810 23:02:41.146730 11325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:41.150241 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11187
2026-08-10T23:02:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:41 dist-test-slave-rwrg krb5kdc[11359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:41 dist-test-slave-rwrg krb5kdc[11359](info): set up 2 sockets
Aug 10 23:02:41 dist-test-slave-rwrg krb5kdc[11359](info): commencing operation
krb5kdc: starting...
W20260810 23:02:42.416230 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:42 dist-test-slave-rwrg krb5kdc[11359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:42.535059 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:42.618207 11386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:42.618410 11386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:42.618433 11386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:42.619504 11386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:42.619536 11386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:42.619580 11386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:42.619604 11386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:42.619629 11386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:42.621076 11386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:42.621274 11386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:42.621440 11386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:42.626186 11395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:42.626214 11399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:42.626292 11386 server_base.cc:1034] running on GCE node
W20260810 23:02:42.626215 11393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:42.626654 11386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:42.626943 11386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:42.628118 11386 hybrid_clock.cc:648] HybridClock initialized: now 1786402962628100 us; error 40 us; skew 500 ppm
Aug 10 23:02:42 dist-test-slave-rwrg krb5kdc[11359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:42.630638 11386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:42.631078 11386 webserver.cc:460] Webserver started at http://127.0.0.1:41447/ using document root <none> and password file <none>
I20260810 23:02:42.631338 11386 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:42.631410 11386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:42.631558 11386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:42.632712 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de"
format_stamp: "Formatted at 2026-08-10 23:02:42 on dist-test-slave-rwrg"
server_key: "58c8c298094b1c618ce99876f532f77a"
server_key_iv: "e4eeb462e824b2ec104bde4f4d151ed5"
server_key_version: "encryptionkey@0"
I20260810 23:02:42.633087 11386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de"
format_stamp: "Formatted at 2026-08-10 23:02:42 on dist-test-slave-rwrg"
server_key: "58c8c298094b1c618ce99876f532f77a"
server_key_iv: "e4eeb462e824b2ec104bde4f4d151ed5"
server_key_version: "encryptionkey@0"
I20260810 23:02:42.634517 11386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:42.635229 11408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:42.635416 11386 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:42.635526 11386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de"
format_stamp: "Formatted at 2026-08-10 23:02:42 on dist-test-slave-rwrg"
server_key: "58c8c298094b1c618ce99876f532f77a"
server_key_iv: "e4eeb462e824b2ec104bde4f4d151ed5"
server_key_version: "encryptionkey@0"
I20260810 23:02:42.635614 11386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:42.639015 11386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:42.641029 11386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:42.641187 11386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:42.645296 11386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35593
I20260810 23:02:42.645328 11500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35593 every 8 connection(s)
I20260810 23:02:42.645678 11386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:42.646721 11502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:42.649107 11502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de: Bootstrap starting.
I20260810 23:02:42.649746 11502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:42.650061 11502 log.cc:824] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:42.650187 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11386
I20260810 23:02:42.650322 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:42.650502 6334 external_mini_cluster.cc:1428] Setting key 72e2e8b22361364ba6c3b25cdf18dd50
I20260810 23:02:42.650954 11502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de: No bootstrap required, opened a new log
I20260810 23:02:42.653219 11502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } }
I20260810 23:02:42.653409 11502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:42.653451 11502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb43f03d0284ba9b39d2ec12bd2f1de, State: Initialized, Role: FOLLOWER
I20260810 23:02:42.653594 11502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } }
I20260810 23:02:42.653716 11502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:42.653759 11502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:42.653816 11502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:42.653928 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:42.651035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41517 (local address 127.0.0.1:35593)
0810 23:02:42.651250 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:42.651260 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:42.651294 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:02:42.652285 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:42.652295 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:42.652330 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:42.652484 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:42.653206 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:42.653211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:42.653211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:42.653215 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:42.653263 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:42.653278 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:42.653356 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:42.653411 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 23:02:42.653563 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260810 23:02:42.654495 11502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } }
I20260810 23:02:42.654629 11502 leader_election.cc:302] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cb43f03d0284ba9b39d2ec12bd2f1de; no voters:
I20260810 23:02:42.654838 11502 leader_election.cc:288] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:42.654915 11511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:42.654978 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:42.655102 11502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:42.655117 11511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [term 1 LEADER]: Becoming Leader. State: Replica: 8cb43f03d0284ba9b39d2ec12bd2f1de, State: Running, Role: LEADER
I20260810 23:02:42.655203 11511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } }
I20260810 23:02:42.655511 11513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } } }
I20260810 23:02:42.655594 11513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:42.655499 11511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cb43f03d0284ba9b39d2ec12bd2f1de. Latest consensus state: current_term: 1 leader_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb43f03d0284ba9b39d2ec12bd2f1de" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35593 } } }
I20260810 23:02:42.655807 11511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:42.656714 11522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:42.657675 11522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:42.658005 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:42.656653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56519 (local address 127.0.0.1:35593)
0810 23:02:42.656685 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:42.656687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:42.656945 (+ 258us) server_negotiation.cc:407] Connection header received
0810 23:02:42.657005 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:42.657006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:42.657010 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:42.657056 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:42.657739 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:42.657740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:42.657740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:42.657744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:42.657759 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:42.657764 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:42.657806 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:42.657937 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 23:02:42.657949 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260810 23:02:42.659301 11522 catalog_manager.cc:1347] Generated new cluster ID: cf3c019ea79e474dae7a3637f1b70ef6
I20260810 23:02:42.659348 11522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:42.683604 11522 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:42.683804 11522 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:42.689961 11522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cb43f03d0284ba9b39d2ec12bd2f1de: Generated new TSK 0
I20260810 23:02:42.690160 11522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:42.701843 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11386
2026-08-10T23:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:42 dist-test-slave-rwrg krb5kdc[11559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:42 dist-test-slave-rwrg krb5kdc[11559](info): set up 2 sockets
Aug 10 23:02:42 dist-test-slave-rwrg krb5kdc[11559](info): commencing operation
krb5kdc: starting...
W20260810 23:02:43.994100 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:44.119302 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:44.202430 11578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:44.202656 11578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:44.202692 11578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:44.203796 11578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:44.203835 11578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:44.203850 11578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:44.203861 11578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:44.203874 11578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:44.205412 11578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:44.205622 11578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:44.205809 11578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:44.210884 11585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:44.210920 11583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:44.210963 11578 server_base.cc:1034] running on GCE node
W20260810 23:02:44.210924 11589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:44.211359 11578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:44.211714 11578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:44.212890 11578 hybrid_clock.cc:648] HybridClock initialized: now 1786402964212867 us; error 43 us; skew 500 ppm
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:44.215847 11578 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:44.216360 11578 webserver.cc:460] Webserver started at http://127.0.0.1:44497/ using document root <none> and password file <none>
I20260810 23:02:44.216650 11578 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:44.216722 11578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:44.216871 11578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:44.217911 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "958bb36612e048a1860267eb598b863b"
format_stamp: "Formatted at 2026-08-10 23:02:44 on dist-test-slave-rwrg"
server_key: "bf3f290c4727aa8b2d4cb63618f53b92"
server_key_iv: "eaa82f2a39c8d274106faa170c248d6d"
server_key_version: "encryptionkey@0"
I20260810 23:02:44.218278 11578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "958bb36612e048a1860267eb598b863b"
format_stamp: "Formatted at 2026-08-10 23:02:44 on dist-test-slave-rwrg"
server_key: "bf3f290c4727aa8b2d4cb63618f53b92"
server_key_iv: "eaa82f2a39c8d274106faa170c248d6d"
server_key_version: "encryptionkey@0"
I20260810 23:02:44.219646 11578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:44.220358 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:44.220571 11578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:44.220669 11578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "958bb36612e048a1860267eb598b863b"
format_stamp: "Formatted at 2026-08-10 23:02:44 on dist-test-slave-rwrg"
server_key: "bf3f290c4727aa8b2d4cb63618f53b92"
server_key_iv: "eaa82f2a39c8d274106faa170c248d6d"
server_key_version: "encryptionkey@0"
I20260810 23:02:44.220745 11578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:44.229965 11578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:44.232203 11578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:44.232358 11578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:44.236469 11578 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43199
I20260810 23:02:44.236543 11691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43199 every 8 connection(s)
I20260810 23:02:44.236905 11578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:44.237900 11693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:44.240211 11693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b: Bootstrap starting.
I20260810 23:02:44.240900 11693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:44.241232 11693 log.cc:824] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:44.241919 11693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b: No bootstrap required, opened a new log
I20260810 23:02:44.243536 11693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260810 23:02:44.243690 11693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:44.243713 11693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 958bb36612e048a1860267eb598b863b, State: Initialized, Role: FOLLOWER
I20260810 23:02:44.243932 11693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260810 23:02:44.244017 11693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:44.244058 11693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:44.244112 11693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:44.244275 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11578
I20260810 23:02:44.244402 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:44.244618 6334 external_mini_cluster.cc:1428] Setting key 951503266d0d80a107669c1c32df11b8
I20260810 23:02:44.244773 11693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260810 23:02:44.244884 11693 leader_election.cc:302] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 958bb36612e048a1860267eb598b863b; no voters:
I20260810 23:02:44.245066 11693 leader_election.cc:288] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:44.245128 11698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:44.245376 11698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [term 1 LEADER]: Becoming Leader. State: Replica: 958bb36612e048a1860267eb598b863b, State: Running, Role: LEADER
I20260810 23:02:44.245397 11693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:44.245486 11698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } }
I20260810 23:02:44.245779 11698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "958bb36612e048a1860267eb598b863b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } } }
I20260810 23:02:44.245853 11698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:44.245975 11698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 958bb36612e048a1860267eb598b863b. Latest consensus state: current_term: 1 leader_uuid: "958bb36612e048a1860267eb598b863b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "958bb36612e048a1860267eb598b863b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43199 } } }
I20260810 23:02:44.246039 11698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:44.246173 11711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:44.246767 11711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:44.248524 11711 catalog_manager.cc:1347] Generated new cluster ID: c998c829014246afbec957c5b4f10a9a
I20260810 23:02:44.248634 11711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:02:44.256914 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:44.245251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:34767 (local address 127.0.0.1:43199)
0810 23:02:44.246159 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:44.246181 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:44.246218 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:02:44.246335 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:44.246342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:44.246396 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:44.246533 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:44.247294 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:44.248453 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:44.249614 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:44.250203 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:44.250598 (+ 395us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:44.253026 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:44.253030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:44.253030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:44.253041 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:44.254928 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:44.255348 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:44.255351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:44.255351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:44.255457 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:44.255666 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:44.255667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:44.255667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:44.255939 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:44.256509 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:44.256590 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 23:02:44.256690 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":58,"threads_started":1}
I20260810 23:02:44.257603 11711 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:44.258010 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:44.258229 11711 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:44.264099 11711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 958bb36612e048a1860267eb598b863b: Generated new TSK 0
I20260810 23:02:44.264307 11711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:44.268182 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:44.259513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:47671 (local address 127.0.0.1:43199)
0810 23:02:44.259550 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:44.259551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:44.260726 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 23:02:44.260805 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:44.260806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:44.260817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:44.260875 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:44.261454 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:44.262244 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:44.263218 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:44.263534 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:44.263571 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:44.265516 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:44.265517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:44.265517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:44.265524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:44.266912 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:44.267294 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:44.267295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:44.267295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:44.267355 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:44.267611 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:44.267612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:44.267612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:44.267718 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:44.267825 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:44.267950 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 23:02:44.268013 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:02:44.270066 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11578
W20260810 23:02:44.285727 11557 connection.cc:535] client connection to 127.0.0.1:43199 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43199): Connection reset by peer (error 104)
2026-08-10T23:02:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11748](info): set up 2 sockets
Aug 10 23:02:44 dist-test-slave-rwrg krb5kdc[11748](info): commencing operation
krb5kdc: starting...
W20260810 23:02:45.562678 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:45 dist-test-slave-rwrg krb5kdc[11748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:45.701375 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:45.787545 11773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:45.787765 11773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:45.787789 11773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:45.789033 11773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:45.789079 11773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:45.789094 11773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:45.789108 11773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:45.789153 11773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:45.790635 11773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:45.790904 11773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:45.791142 11773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:45.796156 11781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:45.796162 11779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:45.796162 11783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:45.796427 11773 server_base.cc:1034] running on GCE node
I20260810 23:02:45.796728 11773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:45.797021 11773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:45.798171 11773 hybrid_clock.cc:648] HybridClock initialized: now 1786402965798137 us; error 46 us; skew 500 ppm
Aug 10 23:02:45 dist-test-slave-rwrg krb5kdc[11748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:45.800733 11773 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:45.801127 11773 webserver.cc:460] Webserver started at http://127.0.0.1:41391/ using document root <none> and password file <none>
I20260810 23:02:45.801309 11773 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:45.801348 11773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:45.801434 11773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:45.802434 11773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "9c3111bcda8f421e87c8b84a160614a5"
format_stamp: "Formatted at 2026-08-10 23:02:45 on dist-test-slave-rwrg"
server_key: "308ab599977a3eb4586ece86af5fcd7c"
server_key_iv: "5878930c671f4429791cf9c5da371ed0"
server_key_version: "encryptionkey@0"
I20260810 23:02:45.802767 11773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "9c3111bcda8f421e87c8b84a160614a5"
format_stamp: "Formatted at 2026-08-10 23:02:45 on dist-test-slave-rwrg"
server_key: "308ab599977a3eb4586ece86af5fcd7c"
server_key_iv: "5878930c671f4429791cf9c5da371ed0"
server_key_version: "encryptionkey@0"
I20260810 23:02:45.804005 11773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:45.804790 11799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:45.804953 11773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:45.805065 11773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "9c3111bcda8f421e87c8b84a160614a5"
format_stamp: "Formatted at 2026-08-10 23:02:45 on dist-test-slave-rwrg"
server_key: "308ab599977a3eb4586ece86af5fcd7c"
server_key_iv: "5878930c671f4429791cf9c5da371ed0"
server_key_version: "encryptionkey@0"
I20260810 23:02:45.805159 11773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:45.819010 11773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:45.820886 11773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:45.821045 11773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:45.825145 11773 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34733
I20260810 23:02:45.825182 11893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34733 every 8 connection(s)
I20260810 23:02:45.825536 11773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:45.826566 11896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:45.826947 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11773
I20260810 23:02:45.827071 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:45.827289 6334 external_mini_cluster.cc:1428] Setting key 1aa09fb3bd50149e7244e4ac8575e756
I20260810 23:02:45.829437 11896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5: Bootstrap starting.
I20260810 23:02:45.830063 11896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:45.830353 11896 log.cc:824] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:45.831106 11896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5: No bootstrap required, opened a new log
I20260810 23:02:45.833122 11896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260810 23:02:45.833331 11896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:45.833398 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:45.827896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:60745 (local address 127.0.0.1:34733)
0810 23:02:45.828109 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:45.828120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:45.828157 (+ 37us) server_negotiation.cc:407] Connection header received
0810 23:02:45.828730 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:45.828744 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:45.828802 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:45.828976 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:45.829225 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:45.830269 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:45.831291 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:45.831724 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:45.831853 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:45.832715 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:45.832719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:45.832719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:45.832896 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:45.832925 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:45.832936 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:45.833034 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:45.833168 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 23:02:45.833199 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":84,"threads_started":1}
I20260810 23:02:45.833400 11896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3111bcda8f421e87c8b84a160614a5, State: Initialized, Role: FOLLOWER
I20260810 23:02:45.833600 11896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260810 23:02:45.833688 11896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:45.833711 11896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:45.833778 11896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:45.834451 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:45.834556 11896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260810 23:02:45.834708 11896 leader_election.cc:302] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c3111bcda8f421e87c8b84a160614a5; no voters:
I20260810 23:02:45.834920 11896 leader_election.cc:288] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:45.834954 11902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:45.835074 11902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9c3111bcda8f421e87c8b84a160614a5, State: Running, Role: LEADER
I20260810 23:02:45.835176 11902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } }
I20260810 23:02:45.835177 11896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:45.835563 11904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c3111bcda8f421e87c8b84a160614a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } } }
I20260810 23:02:45.835642 11904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:45.835870 11912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:45.836441 11912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:45.836928 11903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c3111bcda8f421e87c8b84a160614a5. Latest consensus state: current_term: 1 leader_uuid: "9c3111bcda8f421e87c8b84a160614a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c3111bcda8f421e87c8b84a160614a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34733 } } }
I20260810 23:02:45.837015 11903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:45.838243 11912 catalog_manager.cc:1347] Generated new cluster ID: dedc15f4bc9f4c78971d289c0d428943
I20260810 23:02:45.838305 11912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:45.841080 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:45.837457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:57593 (local address 127.0.0.1:34733)
0810 23:02:45.837496 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:45.837498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:45.837641 (+ 143us) server_negotiation.cc:407] Connection header received
0810 23:02:45.837736 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:45.837737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:45.837746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:45.837775 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:45.837973 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:45.838688 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:45.839709 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:45.840022 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:45.840095 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:45.840811 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:45.840812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:45.840812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:45.840816 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:45.840831 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:45.840834 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:45.840891 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:45.841000 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 23:02:45.841011 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:02:45.850600 11912 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:45.851128 11912 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:45.860381 11912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c3111bcda8f421e87c8b84a160614a5: Generated new TSK 0
I20260810 23:02:45.860579 11912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:45.864113 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11773
W20260810 23:02:45.877748 11744 connection.cc:535] client connection to 127.0.0.1:34733 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34733): Connection reset by peer (error 104)
2026-08-10T23:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:45 dist-test-slave-rwrg krb5kdc[11955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:45 dist-test-slave-rwrg krb5kdc[11955](info): set up 2 sockets
Aug 10 23:02:45 dist-test-slave-rwrg krb5kdc[11955](info): commencing operation
krb5kdc: starting...
W20260810 23:02:47.158252 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:47 dist-test-slave-rwrg krb5kdc[11955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:47.283072 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34953
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:47.366631 11977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:47.366834 11977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:47.366858 11977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:47.368103 11977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:47.368149 11977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:47.368162 11977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:47.368176 11977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:47.368213 11977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:47.369760 11977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:47.370028 11977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:47.370262 11977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:47.375285 11984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:47.375423 11986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:47.375517 11988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:47.375649 11977 server_base.cc:1034] running on GCE node
I20260810 23:02:47.376015 11977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:47.376267 11977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:47.377471 11977 hybrid_clock.cc:648] HybridClock initialized: now 1786402967377458 us; error 42 us; skew 500 ppm
Aug 10 23:02:47 dist-test-slave-rwrg krb5kdc[11955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:47.380071 11977 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:47.380611 11977 webserver.cc:460] Webserver started at http://127.0.0.1:42865/ using document root <none> and password file <none>
I20260810 23:02:47.380846 11977 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:47.380893 11977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:47.381039 11977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:47.382122 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "3297f1650b884aea9b2bf4a89890407d"
format_stamp: "Formatted at 2026-08-10 23:02:47 on dist-test-slave-rwrg"
server_key: "577a66d6a676b84d8f188c9a1a8400dc"
server_key_iv: "a17aaed831c7adf87c546eb7887479ff"
server_key_version: "encryptionkey@0"
I20260810 23:02:47.382496 11977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "3297f1650b884aea9b2bf4a89890407d"
format_stamp: "Formatted at 2026-08-10 23:02:47 on dist-test-slave-rwrg"
server_key: "577a66d6a676b84d8f188c9a1a8400dc"
server_key_iv: "a17aaed831c7adf87c546eb7887479ff"
server_key_version: "encryptionkey@0"
I20260810 23:02:47.383791 11977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:47.384640 12004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:47.384802 11977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:47.384897 11977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "3297f1650b884aea9b2bf4a89890407d"
format_stamp: "Formatted at 2026-08-10 23:02:47 on dist-test-slave-rwrg"
server_key: "577a66d6a676b84d8f188c9a1a8400dc"
server_key_iv: "a17aaed831c7adf87c546eb7887479ff"
server_key_version: "encryptionkey@0"
I20260810 23:02:47.384953 11977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:47.388167 11977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:47.390555 11977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:47.390712 11977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:47.394737 11977 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34953
I20260810 23:02:47.394794 12108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34953 every 8 connection(s)
I20260810 23:02:47.395133 11977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:47.396174 12109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:47.398432 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 11977
I20260810 23:02:47.398569 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:47.398584 12109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d: Bootstrap starting.
I20260810 23:02:47.398751 6334 external_mini_cluster.cc:1428] Setting key 7d504cfc8c5c9267a532a6b030ae2af6
I20260810 23:02:47.399443 12109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:47.399777 12109 log.cc:824] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:47.400612 12109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d: No bootstrap required, opened a new log
I20260810 23:02:47.401506 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:47.399289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:54327 (local address 127.0.0.1:34953)
0810 23:02:47.399511 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:47.399520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:47.399556 (+ 36us) server_negotiation.cc:407] Connection header received
0810 23:02:47.400150 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:47.400158 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:47.400185 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:47.400371 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:47.401078 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:47.401082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:47.401082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:47.401087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:47.401132 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:47.401146 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:47.401237 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:47.401272 (+ 35us) server_negotiation.cc:299] Negotiation successful
0810 23:02:47.401315 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260810 23:02:47.402549 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:47.403046 12109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } }
I20260810 23:02:47.403187 12109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:47.403210 12109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3297f1650b884aea9b2bf4a89890407d, State: Initialized, Role: FOLLOWER
I20260810 23:02:47.403344 12109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } }
I20260810 23:02:47.403429 12109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:47.403472 12109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:47.403522 12109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:47.404202 12109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } }
I20260810 23:02:47.404325 12109 leader_election.cc:302] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3297f1650b884aea9b2bf4a89890407d; no voters:
I20260810 23:02:47.404520 12109 leader_election.cc:288] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:47.404563 12126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:47.404764 12126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [term 1 LEADER]: Becoming Leader. State: Replica: 3297f1650b884aea9b2bf4a89890407d, State: Running, Role: LEADER
I20260810 23:02:47.404795 12109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:47.404898 12126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } }
I20260810 23:02:47.405422 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:47.404043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:50759 (local address 127.0.0.1:34953)
0810 23:02:47.404082 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:47.404084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:47.404205 (+ 121us) server_negotiation.cc:407] Connection header received
0810 23:02:47.404248 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:47.404249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:47.404253 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:47.404309 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:47.405170 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:47.405171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:47.405171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:47.405175 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:47.405190 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:47.405194 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:47.405253 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:47.405327 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 23:02:47.405343 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:02:47.405679 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3297f1650b884aea9b2bf4a89890407d. Latest consensus state: current_term: 1 leader_uuid: "3297f1650b884aea9b2bf4a89890407d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } } }
I20260810 23:02:47.405825 12127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3297f1650b884aea9b2bf4a89890407d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3297f1650b884aea9b2bf4a89890407d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34953 } } }
I20260810 23:02:47.405925 12127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:47.405850 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:47.406141 12144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:47.406713 12144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:47.408134 12144 catalog_manager.cc:1347] Generated new cluster ID: 9bb9d5a8dfae4ae68ce7679801da0c8f
I20260810 23:02:47.408187 12144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:47.415710 12144 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:47.415864 12144 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:47.419498 12144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3297f1650b884aea9b2bf4a89890407d: Generated new TSK 0
I20260810 23:02:47.419723 12144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:47.426329 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 11977
2026-08-10T23:02:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:47 dist-test-slave-rwrg krb5kdc[12159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:47 dist-test-slave-rwrg krb5kdc[12159](info): set up 2 sockets
Aug 10 23:02:47 dist-test-slave-rwrg krb5kdc[12159](info): commencing operation
krb5kdc: starting...
W20260810 23:02:48.710803 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:48 dist-test-slave-rwrg krb5kdc[12159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:48.839228 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:48.923086 12185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:48.923329 12185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:48.923368 12185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:48.924468 12185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:48.924576 12185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:48.924604 12185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:48.924629 12185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:48.924652 12185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:48.926121 12185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40457
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:48.926404 12185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:48.926633 12185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:48.931546 12195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:48.931535 12200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:48.931535 12196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:48.931818 12185 server_base.cc:1034] running on GCE node
I20260810 23:02:48.932085 12185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:48.932364 12185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:48.933508 12185 hybrid_clock.cc:648] HybridClock initialized: now 1786402968933515 us; error 50 us; skew 500 ppm
Aug 10 23:02:48 dist-test-slave-rwrg krb5kdc[12159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:48.936090 12185 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:48.936566 12185 webserver.cc:460] Webserver started at http://127.0.0.1:46061/ using document root <none> and password file <none>
I20260810 23:02:48.936796 12185 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:48.936838 12185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:48.936929 12185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:48.937975 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "5b4b974e1670448986cfda61c5425b07"
format_stamp: "Formatted at 2026-08-10 23:02:48 on dist-test-slave-rwrg"
server_key: "6d374fd3c69b4aed006f97c0bab18e3e"
server_key_iv: "a9fe69cb90fbfce83d33e58f9f03552b"
server_key_version: "encryptionkey@0"
I20260810 23:02:48.938289 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "5b4b974e1670448986cfda61c5425b07"
format_stamp: "Formatted at 2026-08-10 23:02:48 on dist-test-slave-rwrg"
server_key: "6d374fd3c69b4aed006f97c0bab18e3e"
server_key_iv: "a9fe69cb90fbfce83d33e58f9f03552b"
server_key_version: "encryptionkey@0"
I20260810 23:02:48.939581 12185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:48.940282 12213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:48.940474 12185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:48.940641 12185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "5b4b974e1670448986cfda61c5425b07"
format_stamp: "Formatted at 2026-08-10 23:02:48 on dist-test-slave-rwrg"
server_key: "6d374fd3c69b4aed006f97c0bab18e3e"
server_key_iv: "a9fe69cb90fbfce83d33e58f9f03552b"
server_key_version: "encryptionkey@0"
I20260810 23:02:48.940723 12185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:48.952447 12185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:48.954916 12185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:48.955029 12185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:48.959039 12185 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40457
I20260810 23:02:48.959100 12299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40457 every 8 connection(s)
I20260810 23:02:48.959432 12185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:48.960435 12301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:48.962688 12301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07: Bootstrap starting.
I20260810 23:02:48.963279 12301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:48.963572 12301 log.cc:824] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:48.964197 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12185
I20260810 23:02:48.964310 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:48.964392 12301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07: No bootstrap required, opened a new log
I20260810 23:02:48.964537 6334 external_mini_cluster.cc:1428] Setting key 471d65f9ecb160c72a45bdea909ba414
I20260810 23:02:48.966866 12301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } }
I20260810 23:02:48.966996 12301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:48.967018 12301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b4b974e1670448986cfda61c5425b07, State: Initialized, Role: FOLLOWER
I20260810 23:02:48.967170 12301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } }
I20260810 23:02:48.967242 12301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:48.967262 12301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:48.967332 12301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:48.967958 12301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } }
I20260810 23:02:48.968045 12301 leader_election.cc:302] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b4b974e1670448986cfda61c5425b07; no voters:
I20260810 23:02:48.968279 12301 leader_election.cc:288] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:48.968305 12308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:48.968556 12308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [term 1 LEADER]: Becoming Leader. State: Replica: 5b4b974e1670448986cfda61c5425b07, State: Running, Role: LEADER
I20260810 23:02:48.968603 12301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:48.968734 12308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } }
I20260810 23:02:48.969159 12308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b4b974e1670448986cfda61c5425b07. Latest consensus state: current_term: 1 leader_uuid: "5b4b974e1670448986cfda61c5425b07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } } }
I20260810 23:02:48.969406 12308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:48.969326 12310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b4b974e1670448986cfda61c5425b07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b4b974e1670448986cfda61c5425b07" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40457 } } }
I20260810 23:02:48.969851 12310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:48.969851 12329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:48.970577 12329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:48.972124 12329 catalog_manager.cc:1347] Generated new cluster ID: c7bede3ca10f4b45a7e49ce3687e5298
I20260810 23:02:48.972191 12329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:48 dist-test-slave-rwrg krb5kdc[12159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 23:02:48.976226 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:48.965049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:42459 (local address 127.0.0.1:40457)
0810 23:02:48.965237 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:48.965244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:48.965270 (+ 26us) server_negotiation.cc:407] Connection header received
0810 23:02:48.965809 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:48.965817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:48.965865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:48.965975 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:48.966750 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:48.968088 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:48.969217 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:48.969922 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:48.970014 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:48.972982 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:48.972986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:48.972987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:48.972998 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:48.974655 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:48.975049 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:48.975052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:48.975053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:48.975141 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:48.975384 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:48.975386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:48.975386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:48.975613 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:48.975827 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:48.975928 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 23:02:48.976029 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260810 23:02:48.977500 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:48.984997 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:48.978891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:39839 (local address 127.0.0.1:40457)
0810 23:02:48.978926 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:48.978928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:48.979032 (+ 104us) server_negotiation.cc:407] Connection header received
0810 23:02:48.979104 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:48.979105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:48.979116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:48.979170 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:48.979660 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:48.980160 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:48.980957 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:48.981162 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:48.981250 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:48.982296 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:48.982297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:48.982298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:48.982304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:48.983745 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:48.984114 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:48.984116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:48.984116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:48.984175 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:48.984402 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:48.984403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:48.984404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:48.984573 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:48.984686 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:48.984811 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 23:02:48.984874 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:02:48.989416 12329 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:48.989920 12329 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:48.994771 12329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b4b974e1670448986cfda61c5425b07: Generated new TSK 0
I20260810 23:02:48.994949 12329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:48.999948 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12185
W20260810 23:02:49.014365 12156 connection.cc:535] client connection to 127.0.0.1:40457 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40457): Connection reset by peer (error 104)
2026-08-10T23:02:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:49 dist-test-slave-rwrg krb5kdc[12358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:49 dist-test-slave-rwrg krb5kdc[12358](info): set up 2 sockets
Aug 10 23:02:49 dist-test-slave-rwrg krb5kdc[12358](info): commencing operation
krb5kdc: starting...
W20260810 23:02:50.349028 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:50 dist-test-slave-rwrg krb5kdc[12358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:50.475435 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35337
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:50.558807 12379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:50.559010 12379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:50.559039 12379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:50.560109 12379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:50.560145 12379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:50.560159 12379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:50.560171 12379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:50.560184 12379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:50.561641 12379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:50.561837 12379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:50.562005 12379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:02:50.567067 12379 server_base.cc:1034] running on GCE node
W20260810 23:02:50.567039 12388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:50.567031 12387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:50.567039 12393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:50.567565 12379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:50.567839 12379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:50.569025 12379 hybrid_clock.cc:648] HybridClock initialized: now 1786402970569003 us; error 37 us; skew 500 ppm
Aug 10 23:02:50 dist-test-slave-rwrg krb5kdc[12358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:50.576424 12379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:50.576942 12379 webserver.cc:460] Webserver started at http://127.0.0.1:45793/ using document root <none> and password file <none>
I20260810 23:02:50.577190 12379 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:50.577260 12379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:50.577411 12379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:50.578459 12379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "49aaffd0a11b48bbb91ed5877d620e8c"
format_stamp: "Formatted at 2026-08-10 23:02:50 on dist-test-slave-rwrg"
server_key: "2c3a11753b3abca8bb1d63286f26c252"
server_key_iv: "9aa82df9276ea599be7341e80837ec5f"
server_key_version: "encryptionkey@0"
I20260810 23:02:50.578907 12379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "49aaffd0a11b48bbb91ed5877d620e8c"
format_stamp: "Formatted at 2026-08-10 23:02:50 on dist-test-slave-rwrg"
server_key: "2c3a11753b3abca8bb1d63286f26c252"
server_key_iv: "9aa82df9276ea599be7341e80837ec5f"
server_key_version: "encryptionkey@0"
I20260810 23:02:50.580295 12379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:50.581205 12401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:50.581431 12379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:02:50.581548 12379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "49aaffd0a11b48bbb91ed5877d620e8c"
format_stamp: "Formatted at 2026-08-10 23:02:50 on dist-test-slave-rwrg"
server_key: "2c3a11753b3abca8bb1d63286f26c252"
server_key_iv: "9aa82df9276ea599be7341e80837ec5f"
server_key_version: "encryptionkey@0"
I20260810 23:02:50.581645 12379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:50.617493 12379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:50.619465 12379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:50.619652 12379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:50.623926 12379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35337
I20260810 23:02:50.623976 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35337 every 8 connection(s)
I20260810 23:02:50.624393 12379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:50.625306 12502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:50.627651 12502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: Bootstrap starting.
I20260810 23:02:50.628340 12502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:50.628717 12502 log.cc:824] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:50.629434 12502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: No bootstrap required, opened a new log
I20260810 23:02:50.631029 12502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260810 23:02:50.631213 12502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:50.631269 12502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49aaffd0a11b48bbb91ed5877d620e8c, State: Initialized, Role: FOLLOWER
I20260810 23:02:50.631346 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12379
I20260810 23:02:50.631455 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:50.631435 12502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260810 23:02:50.631523 12502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:50.631565 12502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:50.631618 12502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:50.631635 6334 external_mini_cluster.cc:1428] Setting key 06103b5f1110968291374902450ce878
I20260810 23:02:50.632295 12502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260810 23:02:50.632411 12502 leader_election.cc:302] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49aaffd0a11b48bbb91ed5877d620e8c; no voters:
I20260810 23:02:50.632613 12502 leader_election.cc:288] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:50.632632 12510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:50.632747 12510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [term 1 LEADER]: Becoming Leader. State: Replica: 49aaffd0a11b48bbb91ed5877d620e8c, State: Running, Role: LEADER
I20260810 23:02:50.632892 12502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:50.632900 12510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } }
I20260810 23:02:50.633432 12516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } } }
I20260810 23:02:50.633567 12516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:50.633926 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49aaffd0a11b48bbb91ed5877d620e8c. Latest consensus state: current_term: 1 leader_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49aaffd0a11b48bbb91ed5877d620e8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35337 } } }
I20260810 23:02:50.634120 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c [sys.catalog]: This master's current role is: LEADER
W20260810 23:02:50.634822 12545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:02:50.634881 12545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:02:50.634928 12533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:50.635079 12533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:50.636550 12533 catalog_manager.cc:1347] Generated new cluster ID: ff1e5c7f40bd42c7966fd029c83757ff
I20260810 23:02:50.636605 12533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:50.637657 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:50.632410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:39283 (local address 127.0.0.1:35337)
0810 23:02:50.632618 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:50.632627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:50.632683 (+ 56us) server_negotiation.cc:407] Connection header received
0810 23:02:50.632805 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:50.632812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:50.632866 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:50.633281 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:50.633375 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:50.634762 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:50.635672 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:50.636248 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:50.636391 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:50.637027 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:50.637031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:50.637032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:50.637236 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:50.637265 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:50.637276 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:50.637373 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:50.637459 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:02:50.637487 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":127,"threads_started":1}
I20260810 23:02:50.638447 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:50.642645 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:50.639857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:40481 (local address 127.0.0.1:35337)
0810 23:02:50.639895 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:50.639896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:50.639970 (+ 74us) server_negotiation.cc:407] Connection header received
0810 23:02:50.640042 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:50.640042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:50.640050 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:50.640093 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:50.640210 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:50.640706 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:50.641420 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:50.641643 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:50.641722 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:50.642395 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:50.642397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:50.642397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:50.642402 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:50.642415 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:50.642419 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:50.642473 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:50.642545 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 23:02:50.642557 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:02:50.651597 12533 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:50.652307 12533 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:50.660570 12533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49aaffd0a11b48bbb91ed5877d620e8c: Generated new TSK 0
I20260810 23:02:50.660750 12533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:50.670027 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12379
W20260810 23:02:50.685451 12355 connection.cc:535] client connection to 127.0.0.1:35337 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35337): Connection reset by peer (error 104)
2026-08-10T23:02:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:50 dist-test-slave-rwrg krb5kdc[12569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:50 dist-test-slave-rwrg krb5kdc[12569](info): set up 2 sockets
Aug 10 23:02:50 dist-test-slave-rwrg krb5kdc[12569](info): commencing operation
krb5kdc: starting...
W20260810 23:02:51.953459 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:52 dist-test-slave-rwrg krb5kdc[12569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 23:02:52.081262 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35321
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:52.163959 12589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:52.164206 12589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:52.164247 12589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:52.165405 12589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:52.165477 12589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:52.165503 12589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:52.165529 12589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:52.165555 12589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:52.167038 12589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:52.167322 12589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:52.167547 12589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:52.172384 12602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:52.172415 12600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:52.172447 12598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:52.172659 12589 server_base.cc:1034] running on GCE node
I20260810 23:02:52.172981 12589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:52.173250 12589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:52.174459 12589 hybrid_clock.cc:648] HybridClock initialized: now 1786402972174442 us; error 44 us; skew 500 ppm
Aug 10 23:02:52 dist-test-slave-rwrg krb5kdc[12569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402972, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:52.177618 12589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 23:02:52.178153 12589 webserver.cc:460] Webserver started at http://127.0.0.1:39925/ using document root <none> and password file <none>
I20260810 23:02:52.178416 12589 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:52.178500 12589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:52.178668 12589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:52.179821 12589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "8a097d7b8d574639878700cd289a8026"
format_stamp: "Formatted at 2026-08-10 23:02:52 on dist-test-slave-rwrg"
server_key: "d77fb202aa6fe04f33a99778efd202e8"
server_key_iv: "91a3447163d15e60a4008172ab2028c4"
server_key_version: "encryptionkey@0"
I20260810 23:02:52.180230 12589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "8a097d7b8d574639878700cd289a8026"
format_stamp: "Formatted at 2026-08-10 23:02:52 on dist-test-slave-rwrg"
server_key: "d77fb202aa6fe04f33a99778efd202e8"
server_key_iv: "91a3447163d15e60a4008172ab2028c4"
server_key_version: "encryptionkey@0"
I20260810 23:02:52.181708 12589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:52.182476 12610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:52.182667 12589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:52.182771 12589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "8a097d7b8d574639878700cd289a8026"
format_stamp: "Formatted at 2026-08-10 23:02:52 on dist-test-slave-rwrg"
server_key: "d77fb202aa6fe04f33a99778efd202e8"
server_key_iv: "91a3447163d15e60a4008172ab2028c4"
server_key_version: "encryptionkey@0"
I20260810 23:02:52.182861 12589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:52.186396 12589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:52.188901 12589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:52.189126 12589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:52.193662 12589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35321
I20260810 23:02:52.193724 12707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35321 every 8 connection(s)
I20260810 23:02:52.194079 12589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:52.195080 12708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:52.196837 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12589
I20260810 23:02:52.196959 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:52.197136 6334 external_mini_cluster.cc:1428] Setting key fd5598288045ca651983bd52c5f828c2
I20260810 23:02:52.197810 12708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026: Bootstrap starting.
I20260810 23:02:52.198485 12708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:52.198870 12708 log.cc:824] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:52.199730 12708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026: No bootstrap required, opened a new log
I20260810 23:02:52.199910 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:52.197754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:58445 (local address 127.0.0.1:35321)
0810 23:02:52.198007 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:52.198016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:52.198046 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:02:52.198518 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:52.198550 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:52.198583 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:52.198714 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:52.199431 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:52.199435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:52.199435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:52.199440 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:52.199485 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:52.199499 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:52.199596 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:52.199634 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 23:02:52.199685 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260810 23:02:52.200969 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:52.202339 12708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } }
I20260810 23:02:52.202560 12708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:52.202633 12708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a097d7b8d574639878700cd289a8026, State: Initialized, Role: FOLLOWER
I20260810 23:02:52.202771 12708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } }
I20260810 23:02:52.202864 12708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:52.202888 12708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:52.203212 12708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:52.203502 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:52.202413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.237:35129 (local address 127.0.0.1:35321)
0810 23:02:52.202492 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:52.202494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:52.202594 (+ 100us) server_negotiation.cc:407] Connection header received
0810 23:02:52.202646 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:52.202647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:52.202652 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:52.202717 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:52.203280 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:52.203281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:52.203282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:52.203286 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:52.203299 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:52.203303 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:52.203351 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:52.203421 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:02:52.203431 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 23:02:52.204015 12708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } }
I20260810 23:02:52.204140 12708 leader_election.cc:302] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a097d7b8d574639878700cd289a8026; no voters:
I20260810 23:02:52.204319 12708 leader_election.cc:288] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:52.204363 12722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:52.204564 12722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [term 1 LEADER]: Becoming Leader. State: Replica: 8a097d7b8d574639878700cd289a8026, State: Running, Role: LEADER
I20260810 23:02:52.204607 12708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:52.204668 12722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } }
I20260810 23:02:52.205076 12725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a097d7b8d574639878700cd289a8026. Latest consensus state: current_term: 1 leader_uuid: "8a097d7b8d574639878700cd289a8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } } }
I20260810 23:02:52.205076 12724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a097d7b8d574639878700cd289a8026" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a097d7b8d574639878700cd289a8026" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35321 } } }
I20260810 23:02:52.205200 12725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:52.205207 12724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:52.205433 12737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:52.205924 12737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:52.207516 12737 catalog_manager.cc:1347] Generated new cluster ID: e52b33af07fc4362a393f4c8618412a8
I20260810 23:02:52.207566 12737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:52.239550 12737 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:52.239770 12737 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:52.246546 12737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a097d7b8d574639878700cd289a8026: Generated new TSK 0
I20260810 23:02:52.246753 12737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:52.258044 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12589
2026-08-10T23:02:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:52 dist-test-slave-rwrg krb5kdc[12760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:52 dist-test-slave-rwrg krb5kdc[12760](info): set up 2 sockets
Aug 10 23:02:52 dist-test-slave-rwrg krb5kdc[12760](info): commencing operation
krb5kdc: starting...
W20260810 23:02:53.536325 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:53.661918 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44287
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:39121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:44287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:53.756625 12786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:53.756840 12786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:53.756878 12786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:53.757954 12786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:53.758018 12786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:53.758044 12786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:53.758072 12786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:53.758095 12786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:53.759501 12786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:39121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:44287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:44287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:53.759780 12786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:53.760007 12786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:53.765673 12796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:53.765739 12797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:53.765673 12808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:53.765844 12786 server_base.cc:1034] running on GCE node
I20260810 23:02:53.766177 12786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:53.766420 12786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:53.767562 12786 hybrid_clock.cc:648] HybridClock initialized: now 1786402973767544 us; error 33 us; skew 500 ppm
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402973, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:53.770099 12786 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:53.770490 12786 webserver.cc:460] Webserver started at http://127.6.47.190:42529/ using document root <none> and password file <none>
I20260810 23:02:53.770689 12786 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:53.770730 12786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:53.770812 12786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:53.771749 12786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "0de519108f0c4eabb6bc82734bbcd96b"
format_stamp: "Formatted at 2026-08-10 23:02:53 on dist-test-slave-rwrg"
server_key: "61e6a96adc37a0bfd633690cc9756392"
server_key_iv: "a8d7bce4f5eb6d6c2a526e38ef3481c1"
server_key_version: "encryptionkey@0"
I20260810 23:02:53.772039 12786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "0de519108f0c4eabb6bc82734bbcd96b"
format_stamp: "Formatted at 2026-08-10 23:02:53 on dist-test-slave-rwrg"
server_key: "61e6a96adc37a0bfd633690cc9756392"
server_key_iv: "a8d7bce4f5eb6d6c2a526e38ef3481c1"
server_key_version: "encryptionkey@0"
I20260810 23:02:53.773337 12786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260810 23:02:53.773988 12823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:53.774161 12786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:53.774272 12786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "0de519108f0c4eabb6bc82734bbcd96b"
format_stamp: "Formatted at 2026-08-10 23:02:53 on dist-test-slave-rwrg"
server_key: "61e6a96adc37a0bfd633690cc9756392"
server_key_iv: "a8d7bce4f5eb6d6c2a526e38ef3481c1"
server_key_version: "encryptionkey@0"
I20260810 23:02:53.774345 12786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:53.790438 12786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:53.792280 12786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:53.792446 12786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:53.796623 12786 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:44287
I20260810 23:02:53.796689 12914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:44287 every 8 connection(s)
I20260810 23:02:53.797072 12786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:53.797690 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 12786
I20260810 23:02:53.797809 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:53.798007 6334 external_mini_cluster.cc:1428] Setting key 4bcc8340f61d8a95fc194326e35f49b8
I20260810 23:02:53.798072 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:53.801594 12916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b: Bootstrap starting.
I20260810 23:02:53.802330 12916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:53.802657 12916 log.cc:824] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:53.803500 12916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b: No bootstrap required, opened a new log
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:53.805797 12916 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } }
I20260810 23:02:53.805933 12916 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:53.805956 12916 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de519108f0c4eabb6bc82734bbcd96b, State: Initialized, Role: FOLLOWER
I20260810 23:02:53.806113 12916 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } }
I20260810 23:02:53.806182 12916 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:53.806205 12916 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:53.806273 12916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:53.806805 12916 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } }
I20260810 23:02:53.806895 12916 leader_election.cc:302] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0de519108f0c4eabb6bc82734bbcd96b; no voters:
I20260810 23:02:53.807150 12923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:53.807119 12916 leader_election.cc:288] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:53.807333 12923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [term 1 LEADER]: Becoming Leader. State: Replica: 0de519108f0c4eabb6bc82734bbcd96b, State: Running, Role: LEADER
I20260810 23:02:53.807478 12916 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:53.807477 12923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } }
I20260810 23:02:53.807987 12925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0de519108f0c4eabb6bc82734bbcd96b. Latest consensus state: current_term: 1 leader_uuid: "0de519108f0c4eabb6bc82734bbcd96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } } }
I20260810 23:02:53.808085 12925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:53.808291 12928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0de519108f0c4eabb6bc82734bbcd96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de519108f0c4eabb6bc82734bbcd96b" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 44287 } } }
I20260810 23:02:53.808326 12942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:53.808372 12928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:53.808871 12942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:53.809876 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:53.798523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55765 (local address 127.6.47.190:44287)
0810 23:02:53.798710 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:53.798720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:53.798754 (+ 34us) server_negotiation.cc:407] Connection header received
0810 23:02:53.799271 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:53.799278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:53.799327 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:53.799440 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:53.800261 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:53.801203 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:53.802255 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:53.802695 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:53.802819 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:53.805496 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:53.805500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:53.805500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:53.805511 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:53.807615 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:53.808095 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:53.808097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:53.808098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:53.808214 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:53.808528 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:53.808529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:53.808529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:53.808859 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:53.809497 (+ 638us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:53.809567 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:02:53.809684 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260810 23:02:53.810973 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:53.812047 12942 catalog_manager.cc:1347] Generated new cluster ID: 59e8ae88a2b147a8b559e82245f3b50c
I20260810 23:02:53.812172 12942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:53.818658 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:53.812575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59271 (local address 127.6.47.190:44287)
0810 23:02:53.812636 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:53.812638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:53.812753 (+ 115us) server_negotiation.cc:407] Connection header received
0810 23:02:53.812819 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:53.812819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:53.812828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:53.812879 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:53.813356 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:53.813832 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:53.814478 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:53.814714 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:53.814789 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:53.815989 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:53.815990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:53.815990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:53.815997 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:53.817453 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:53.817837 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:53.817838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:53.817839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:53.817893 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:53.818137 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:53.818138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:53.818138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:53.818250 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:53.818359 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:53.818486 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 23:02:53.818550 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260810 23:02:53.820226 12942 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:53.820834 12942 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:53.829655 12942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0de519108f0c4eabb6bc82734bbcd96b: Generated new TSK 0
I20260810 23:02:53.829877 12942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:53.835410 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 12786
W20260810 23:02:53.849342 12757 connection.cc:535] client connection to 127.6.47.190:44287 recv error: Network error: failed to read from TLS socket (remote: 127.6.47.190:44287): Connection reset by peer (error 104)
2026-08-10T23:02:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12984](info): set up 2 sockets
Aug 10 23:02:53 dist-test-slave-rwrg krb5kdc[12984](info): commencing operation
krb5kdc: starting...
W20260810 23:02:55.102312 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:55 dist-test-slave-rwrg krb5kdc[12984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:55.223629 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39995
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:44963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:39995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:55.305496 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:55.305747 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:55.305787 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:55.306927 13002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:55.306998 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:55.307022 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:55.307049 13002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:55.307080 13002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:55.308570 13002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:44963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:39995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:39995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:55.308858 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:55.309082 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:55.313936 13013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:55.313915 13021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:55.313974 13017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:55.314060 13002 server_base.cc:1034] running on GCE node
I20260810 23:02:55.314363 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:55.314594 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:55.315758 13002 hybrid_clock.cc:648] HybridClock initialized: now 1786402975315743 us; error 48 us; skew 500 ppm
Aug 10 23:02:55 dist-test-slave-rwrg krb5kdc[12984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402975, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:55.318359 13002 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:55.318815 13002 webserver.cc:460] Webserver started at http://127.6.47.190:44193/ using document root <none> and password file <none>
I20260810 23:02:55.318992 13002 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:55.319031 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:55.319123 13002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:55.320068 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7"
format_stamp: "Formatted at 2026-08-10 23:02:55 on dist-test-slave-rwrg"
server_key: "eae281d1328429ccc1add9d6f18e56e2"
server_key_iv: "a0d39e1b5d37a1403e4d223fdf697969"
server_key_version: "encryptionkey@0"
I20260810 23:02:55.320380 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7"
format_stamp: "Formatted at 2026-08-10 23:02:55 on dist-test-slave-rwrg"
server_key: "eae281d1328429ccc1add9d6f18e56e2"
server_key_iv: "a0d39e1b5d37a1403e4d223fdf697969"
server_key_version: "encryptionkey@0"
I20260810 23:02:55.321676 13002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:55.322348 13034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:55.322522 13002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260810 23:02:55.322618 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7"
format_stamp: "Formatted at 2026-08-10 23:02:55 on dist-test-slave-rwrg"
server_key: "eae281d1328429ccc1add9d6f18e56e2"
server_key_iv: "a0d39e1b5d37a1403e4d223fdf697969"
server_key_version: "encryptionkey@0"
I20260810 23:02:55.322672 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:55.336869 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:55.338814 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:55.338927 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:55.342936 13002 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:39995
I20260810 23:02:55.342994 13126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:39995 every 8 connection(s)
I20260810 23:02:55.343341 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:55.344266 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:55.346681 13129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7: Bootstrap starting.
I20260810 23:02:55.347326 13129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:55.347637 13129 log.cc:824] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:55.348168 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13002
I20260810 23:02:55.348284 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:55.348480 6334 external_mini_cluster.cc:1428] Setting key c0c8abfb18ae03e6eb87f3fcdba47cc8
I20260810 23:02:55.348578 13129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7: No bootstrap required, opened a new log
I20260810 23:02:55.350636 13129 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } }
I20260810 23:02:55.350807 13129 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:55.350857 13129 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f5923f99844139a8ba1e2d8ddc1ec7, State: Initialized, Role: FOLLOWER
I20260810 23:02:55.350994 13129 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } }
I20260810 23:02:55.351080 13129 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:55.351125 13129 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:55.351173 13129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:55.351747 13129 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } }
I20260810 23:02:55.351877 13129 leader_election.cc:302] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7f5923f99844139a8ba1e2d8ddc1ec7; no voters:
I20260810 23:02:55.352072 13129 leader_election.cc:288] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:55.352120 13135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:55.352280 13135 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [term 1 LEADER]: Becoming Leader. State: Replica: e7f5923f99844139a8ba1e2d8ddc1ec7, State: Running, Role: LEADER
I20260810 23:02:55.352350 13129 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:55.352434 13135 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } }
I20260810 23:02:55.352846 13135 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7f5923f99844139a8ba1e2d8ddc1ec7. Latest consensus state: current_term: 1 leader_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } } }
I20260810 23:02:55.352861 13137 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7f5923f99844139a8ba1e2d8ddc1ec7" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 39995 } } }
I20260810 23:02:55.353008 13137 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:55.353317 13135 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:55.353636 13152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:55.354228 13152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:55.355451 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:55.349072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50183 (local address 127.6.47.190:39995)
0810 23:02:55.349307 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:55.349315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:55.349342 (+ 27us) server_negotiation.cc:407] Connection header received
0810 23:02:55.349950 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:55.349958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:55.350010 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:55.350180 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:55.350380 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:55.351723 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:55.352890 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:55.353523 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:55.353611 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:55.354679 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:55.354684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:55.354684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:55.354907 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:55.354945 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:55.354960 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:55.355045 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:55.355221 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 23:02:55.355250 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":101,"threads_started":1}
I20260810 23:02:55.356088 13152 catalog_manager.cc:1347] Generated new cluster ID: 87da928ad0354c50a6324b269bf955f4
I20260810 23:02:55.356159 13152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:55.356562 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:55.367525 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:55.360983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53301 (local address 127.6.47.190:39995)
0810 23:02:55.361024 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:55.361026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:55.364666 (+ 3640us) server_negotiation.cc:407] Connection header received
0810 23:02:55.364777 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:55.364778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:55.364787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:55.364847 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:55.365019 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:55.365499 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:55.366250 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:55.366461 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:55.366537 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:55.367252 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:55.367254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:55.367254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:55.367258 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:55.367279 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:55.367283 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:55.367346 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:55.367412 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 23:02:55.367422 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260810 23:02:55.377806 13152 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:55.378580 13152 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:55.387650 13152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7f5923f99844139a8ba1e2d8ddc1ec7: Generated new TSK 0
I20260810 23:02:55.387850 13152 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:55.390199 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13002
W20260810 23:02:55.395794 12979 connection.cc:535] client connection to 127.6.47.190:39995 recv error: Network error: failed to read from TLS socket (remote: 127.6.47.190:39995): Connection reset by peer (error 104)
2026-08-10T23:02:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:55 dist-test-slave-rwrg krb5kdc[13187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:55 dist-test-slave-rwrg krb5kdc[13187](info): set up 2 sockets
Aug 10 23:02:55 dist-test-slave-rwrg krb5kdc[13187](info): commencing operation
krb5kdc: starting...
W20260810 23:02:56.686009 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:56 dist-test-slave-rwrg krb5kdc[13187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:56.810791 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46441
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:45493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:56.891539 13206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:56.891776 13206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:56.891815 13206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:56.892954 13206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:56.893023 13206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:56.893049 13206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:56.893076 13206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:56.893114 13206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:56.894600 13206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:45493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:46441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:56.894886 13206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:56.895113 13206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:02:56.899906 13214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:56.899923 13212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:56.899941 13216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:56.900251 13206 server_base.cc:1034] running on GCE node
I20260810 23:02:56.900573 13206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:56.900871 13206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:56.902040 13206 hybrid_clock.cc:648] HybridClock initialized: now 1786402976902018 us; error 55 us; skew 500 ppm
Aug 10 23:02:56 dist-test-slave-rwrg krb5kdc[13187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402976, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:56.904629 13206 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:56.905104 13206 webserver.cc:460] Webserver started at http://127.6.47.190:36867/ using document root <none> and password file <none>
I20260810 23:02:56.905331 13206 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:56.905400 13206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:56.905537 13206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:56.906569 13206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "8cb38e3df0e948abbd15c0127c7d37ce"
format_stamp: "Formatted at 2026-08-10 23:02:56 on dist-test-slave-rwrg"
server_key: "61050a6c1cc51c2d28cc41ef274484ea"
server_key_iv: "abf064bc0f4216a6f120c17a38e1530d"
server_key_version: "encryptionkey@0"
I20260810 23:02:56.906949 13206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "8cb38e3df0e948abbd15c0127c7d37ce"
format_stamp: "Formatted at 2026-08-10 23:02:56 on dist-test-slave-rwrg"
server_key: "61050a6c1cc51c2d28cc41ef274484ea"
server_key_iv: "abf064bc0f4216a6f120c17a38e1530d"
server_key_version: "encryptionkey@0"
I20260810 23:02:56.908301 13206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:02:56.909145 13225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:56.909354 13206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 23:02:56.909459 13206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "8cb38e3df0e948abbd15c0127c7d37ce"
format_stamp: "Formatted at 2026-08-10 23:02:56 on dist-test-slave-rwrg"
server_key: "61050a6c1cc51c2d28cc41ef274484ea"
server_key_iv: "abf064bc0f4216a6f120c17a38e1530d"
server_key_version: "encryptionkey@0"
I20260810 23:02:56.909549 13206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:56.912961 13206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:56.915320 13206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:56.915494 13206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:56.919687 13206 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:46441
I20260810 23:02:56.919718 13318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:46441 every 8 connection(s)
I20260810 23:02:56.920046 13206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:56.921154 13322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:56.923409 13322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce: Bootstrap starting.
I20260810 23:02:56.924036 13322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:56.924326 13322 log.cc:824] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:56.925063 13322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce: No bootstrap required, opened a new log
I20260810 23:02:56.925577 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13206
I20260810 23:02:56.925714 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:56.925890 6334 external_mini_cluster.cc:1428] Setting key 4b2f204636ef360702e66bc50d6eaec0
I20260810 23:02:56.927079 13322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } }
I20260810 23:02:56.927251 13322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:56.927294 13322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cb38e3df0e948abbd15c0127c7d37ce, State: Initialized, Role: FOLLOWER
I20260810 23:02:56.927486 13322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } }
I20260810 23:02:56.927604 13322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:56.927649 13322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:56.927749 13322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:56.928380 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:56.926421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37505 (local address 127.6.47.190:46441)
0810 23:02:56.926603 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:56.926612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:56.926637 (+ 25us) server_negotiation.cc:407] Connection header received
0810 23:02:56.927061 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:56.927067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:56.927100 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:56.927217 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:56.927958 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:56.927961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:56.927962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:56.927966 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:56.928017 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:56.928028 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:56.928097 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:56.928175 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 23:02:56.928214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260810 23:02:56.928864 13322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } }
I20260810 23:02:56.928966 13322 leader_election.cc:302] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cb38e3df0e948abbd15c0127c7d37ce; no voters:
I20260810 23:02:56.929126 13322 leader_election.cc:288] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:56.929435 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:56.929458 13322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:56.929162 13332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:56.930922 13332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [term 1 LEADER]: Becoming Leader. State: Replica: 8cb38e3df0e948abbd15c0127c7d37ce, State: Running, Role: LEADER
I20260810 23:02:56.931042 13332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } }
I20260810 23:02:56.931638 13333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } } }
I20260810 23:02:56.931744 13333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:56.932358 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:56.931087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39307 (local address 127.6.47.190:46441)
0810 23:02:56.931145 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:56.931147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:56.931390 (+ 243us) server_negotiation.cc:407] Connection header received
0810 23:02:56.931463 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:56.931463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:56.931469 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:56.931525 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:56.932079 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:56.932080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:56.932081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:02:56.932085 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:56.932101 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:02:56.932113 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:56.932158 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:56.932268 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 23:02:56.932284 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:02:56.932444 13364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:56.932369 13354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cb38e3df0e948abbd15c0127c7d37ce. Latest consensus state: current_term: 1 leader_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cb38e3df0e948abbd15c0127c7d37ce" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 46441 } } }
I20260810 23:02:56.932554 13354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:56.933017 13364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:56.934579 13364 catalog_manager.cc:1347] Generated new cluster ID: 228b0d7b198b4db681fd51358a899b6d
I20260810 23:02:56.934638 13364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:02:56.948935 13364 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:56.949113 13364 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:56.955999 13364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cb38e3df0e948abbd15c0127c7d37ce: Generated new TSK 0
I20260810 23:02:56.956189 13364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:56.961774 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13206
2026-08-10T23:02:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:57 dist-test-slave-rwrg krb5kdc[13376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:57 dist-test-slave-rwrg krb5kdc[13376](info): set up 2 sockets
Aug 10 23:02:57 dist-test-slave-rwrg krb5kdc[13376](info): commencing operation
krb5kdc: starting...
W20260810 23:02:58.244818 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:58.364537 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33045
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:33045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:02:58.449034 13404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:02:58.449415 13404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:02:58.449452 13404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:02:58.450870 13404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:02:58.450924 13404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:02:58.450943 13404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:02:58.450959 13404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:02:58.451040 13404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:02:58.452860 13404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:33045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:33045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:02:58.453169 13404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:02:58.453449 13404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 23:02:58.458788 13404 server_base.cc:1034] running on GCE node
W20260810 23:02:58.458741 13415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:58.458793 13421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:02:58.458753 13416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:02:58.459249 13404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:02:58.459681 13404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:02:58.460814 13404 hybrid_clock.cc:648] HybridClock initialized: now 1786402978460781 us; error 40 us; skew 500 ppm
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402978, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:02:58.463361 13404 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:02:58.463845 13404 webserver.cc:460] Webserver started at http://127.6.47.190:43931/ using document root <none> and password file <none>
I20260810 23:02:58.464084 13404 fs_manager.cc:359] Metadata directory not provided
I20260810 23:02:58.464128 13404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:02:58.464299 13404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:02:58.465368 13404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "9e72b48bdd324f9984a32d0d49f1acc5"
format_stamp: "Formatted at 2026-08-10 23:02:58 on dist-test-slave-rwrg"
server_key: "74614d76b66561b9efbe96b32f75b6c9"
server_key_iv: "9d8f69e87bee42673b438f798dfede95"
server_key_version: "encryptionkey@0"
I20260810 23:02:58.465726 13404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "9e72b48bdd324f9984a32d0d49f1acc5"
format_stamp: "Formatted at 2026-08-10 23:02:58 on dist-test-slave-rwrg"
server_key: "74614d76b66561b9efbe96b32f75b6c9"
server_key_iv: "9d8f69e87bee42673b438f798dfede95"
server_key_version: "encryptionkey@0"
I20260810 23:02:58.467025 13404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:02:58.467734 13433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:02:58.467908 13404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:02:58.468012 13404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "9e72b48bdd324f9984a32d0d49f1acc5"
format_stamp: "Formatted at 2026-08-10 23:02:58 on dist-test-slave-rwrg"
server_key: "74614d76b66561b9efbe96b32f75b6c9"
server_key_iv: "9d8f69e87bee42673b438f798dfede95"
server_key_version: "encryptionkey@0"
I20260810 23:02:58.468094 13404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:02:58.498029 13404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:02:58.499970 13404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:02:58.500149 13404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:02:58.504554 13404 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:33045
I20260810 23:02:58.504607 13521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:33045 every 8 connection(s)
I20260810 23:02:58.504998 13404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:02:58.505903 13525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:02:58.508235 13525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5: Bootstrap starting.
I20260810 23:02:58.508813 13525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5: Neither blocks nor log segments found. Creating new log.
I20260810 23:02:58.509054 13525 log.cc:824] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5: Log is configured to *not* fsync() on all Append() calls
I20260810 23:02:58.509979 13525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5: No bootstrap required, opened a new log
I20260810 23:02:58.511694 13525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } }
I20260810 23:02:58.511834 13525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:02:58.511857 13525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e72b48bdd324f9984a32d0d49f1acc5, State: Initialized, Role: FOLLOWER
I20260810 23:02:58.512022 13525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } }
I20260810 23:02:58.512113 13525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:02:58.512171 13525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:02:58.512223 13525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:02:58.512722 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13404
I20260810 23:02:58.512818 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:02:58.513055 6334 external_mini_cluster.cc:1428] Setting key 5e4b675c9c4f4b93c594bc99055f9ce3
I20260810 23:02:58.513283 13525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } }
I20260810 23:02:58.513409 13525 leader_election.cc:302] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e72b48bdd324f9984a32d0d49f1acc5; no voters:
I20260810 23:02:58.513583 13525 leader_election.cc:288] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:02:58.513629 13530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:02:58.513782 13530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [term 1 LEADER]: Becoming Leader. State: Replica: 9e72b48bdd324f9984a32d0d49f1acc5, State: Running, Role: LEADER
I20260810 23:02:58.513833 13525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:02:58.515271 13530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } }
I20260810 23:02:58.515940 13537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e72b48bdd324f9984a32d0d49f1acc5. Latest consensus state: current_term: 1 leader_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } } }
I20260810 23:02:58.516067 13536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e72b48bdd324f9984a32d0d49f1acc5" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 33045 } } }
I20260810 23:02:58.516108 13537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:58.516116 13536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5 [sys.catalog]: This master's current role is: LEADER
I20260810 23:02:58.516479 13560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:02:58.517025 13560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:02:58.518689 13560 catalog_manager.cc:1347] Generated new cluster ID: e2e04f9572ba4f5bb29ed637a5ea5262
I20260810 23:02:58.518743 13560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:02:58.528364 13560 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:02:58.529073 13560 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:02:58.530483 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:58.513540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35019 (local address 127.6.47.190:33045)
0810 23:02:58.513803 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:58.513812 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:58.513847 (+ 35us) server_negotiation.cc:407] Connection header received
0810 23:02:58.513968 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:58.513974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:58.514028 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:58.514167 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:58.515047 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:58.516539 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:58.517363 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:58.517893 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:58.517999 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:58.526786 (+ 8787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:58.526790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:58.526791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:58.526810 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:58.528802 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:58.529187 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:58.529190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:58.529191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:58.529298 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:58.529519 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:58.529520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:58.529520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:58.529747 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:58.529925 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:58.530026 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 23:02:58.530303 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
I20260810 23:02:58.531366 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:02:58.535620 13560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e72b48bdd324f9984a32d0d49f1acc5: Generated new TSK 0
I20260810 23:02:58.535811 13560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:02:58.541538 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:02:58.532802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54737 (local address 127.6.47.190:33045)
0810 23:02:58.532839 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:02:58.532841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:02:58.535381 (+ 2540us) server_negotiation.cc:407] Connection header received
0810 23:02:58.535477 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:02:58.535478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:02:58.535490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:02:58.535552 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:02:58.536220 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:58.536773 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:58.537544 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:02:58.537762 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:02:58.537837 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:02:58.538992 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:02:58.538994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:02:58.538994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:02:58.539000 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:02:58.540366 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:58.540753 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:58.540754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:58.540754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:58.540815 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:02:58.541021 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:02:58.541021 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:02:58.541022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:02:58.541132 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:02:58.541253 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 23:02:58.541364 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 23:02:58.541443 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:02:58.543361 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13404
2026-08-10T23:02:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13584](info): set up 2 sockets
Aug 10 23:02:58 dist-test-slave-rwrg krb5kdc[13584](info): commencing operation
krb5kdc: starting...
W20260810 23:02:59.841678 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:02:59 dist-test-slave-rwrg krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:02:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:02:59.961555 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:36021
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:36021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:03:00.044373 13611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:00.044690 13611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:00.044739 13611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:00.045908 13611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:00.045981 13611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:00.046007 13611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:00.046037 13611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:00.046068 13611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:00.047576 13611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:36021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:00.047863 13611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:00.048110 13611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:00.053015 13628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:00.053128 13611 server_base.cc:1034] running on GCE node
W20260810 23:03:00.053021 13626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:00.053016 13624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:00.053471 13611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:00.053776 13611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:00.054955 13611 hybrid_clock.cc:648] HybridClock initialized: now 1786402980054935 us; error 38 us; skew 500 ppm
Aug 10 23:03:00 dist-test-slave-rwrg krb5kdc[13584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402980, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:00.057423 13611 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:03:00.057842 13611 webserver.cc:460] Webserver started at http://127.6.47.190:37039/ using document root <none> and password file <none>
I20260810 23:03:00.058071 13611 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:00.058139 13611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:00.058279 13611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:00.059310 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "17f8eabf3c294ba0bc9667a781372a27"
format_stamp: "Formatted at 2026-08-10 23:03:00 on dist-test-slave-rwrg"
server_key: "562cc6c5db129418b2ef2050bd7f7a24"
server_key_iv: "be8783a83a3d5d903083e77f54899dd9"
server_key_version: "encryptionkey@0"
I20260810 23:03:00.059708 13611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "17f8eabf3c294ba0bc9667a781372a27"
format_stamp: "Formatted at 2026-08-10 23:03:00 on dist-test-slave-rwrg"
server_key: "562cc6c5db129418b2ef2050bd7f7a24"
server_key_iv: "be8783a83a3d5d903083e77f54899dd9"
server_key_version: "encryptionkey@0"
I20260810 23:03:00.061131 13611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:03:00.061843 13644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:00.062008 13611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:03:00.062110 13611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "17f8eabf3c294ba0bc9667a781372a27"
format_stamp: "Formatted at 2026-08-10 23:03:00 on dist-test-slave-rwrg"
server_key: "562cc6c5db129418b2ef2050bd7f7a24"
server_key_iv: "be8783a83a3d5d903083e77f54899dd9"
server_key_version: "encryptionkey@0"
I20260810 23:03:00.062179 13611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:00.077441 13611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:00.079281 13611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:00.079455 13611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:00.083578 13611 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:36021
I20260810 23:03:00.083619 13725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:36021 every 8 connection(s)
I20260810 23:03:00.084118 13611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:03:00.085060 13726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:00.086900 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13611
I20260810 23:03:00.087013 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:03:00.087188 6334 external_mini_cluster.cc:1428] Setting key 7c06eceff138be3298c50a7a9755500e
I20260810 23:03:00.087646 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27: Bootstrap starting.
I20260810 23:03:00.088274 13726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:00.088661 13726 log.cc:824] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:00.089432 13726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27: No bootstrap required, opened a new log
I20260810 23:03:00.091770 13726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } }
I20260810 23:03:00.091972 13726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:03:00.092025 13726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f8eabf3c294ba0bc9667a781372a27, State: Initialized, Role: FOLLOWER
I20260810 23:03:00.092186 13726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } }
I20260810 23:03:00.092267 13726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:03:00.092289 13726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:03:00.092342 13726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:00.093062 13726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } }
I20260810 23:03:00.093094 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:00.087739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45327 (local address 127.6.47.190:36021)
0810 23:03:00.087919 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:00.087928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:00.087958 (+ 30us) server_negotiation.cc:407] Connection header received
0810 23:03:00.088424 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:00.088432 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:00.088477 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:00.088642 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:00.088894 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:00.089971 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:00.091142 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:00.091521 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:00.091600 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:00.092402 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:00.092406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:00.092406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:03:00.092613 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:00.092644 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:03:00.092655 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:00.092706 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:00.092814 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 23:03:00.092884 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260810 23:03:00.093153 13726 leader_election.cc:302] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17f8eabf3c294ba0bc9667a781372a27; no voters:
I20260810 23:03:00.093331 13726 leader_election.cc:288] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:03:00.093394 13737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:03:00.093565 13726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:00.093603 13737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [term 1 LEADER]: Becoming Leader. State: Replica: 17f8eabf3c294ba0bc9667a781372a27, State: Running, Role: LEADER
I20260810 23:03:00.093761 13737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } }
I20260810 23:03:00.094139 13739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17f8eabf3c294ba0bc9667a781372a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } } }
I20260810 23:03:00.094246 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:03:00.094256 13739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:00.094834 13749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:03:00.095219 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17f8eabf3c294ba0bc9667a781372a27. Latest consensus state: current_term: 1 leader_uuid: "17f8eabf3c294ba0bc9667a781372a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f8eabf3c294ba0bc9667a781372a27" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 36021 } } }
I20260810 23:03:00.095307 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:00.095396 13749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:03:00.096961 13749 catalog_manager.cc:1347] Generated new cluster ID: 3c459437979843b1bbe001f469c2de50
I20260810 23:03:00.097016 13749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:03:00.100455 13731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:00.096988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52779 (local address 127.6.47.190:36021)
0810 23:03:00.097028 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:00.097030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:00.097304 (+ 274us) server_negotiation.cc:407] Connection header received
0810 23:03:00.097359 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:00.097360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:00.097369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:00.097418 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:00.097556 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:00.098223 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:00.099214 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:00.099527 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:00.099580 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:00.100239 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:00.100240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:00.100241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:03:00.100244 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:00.100258 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:03:00.100262 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:00.100310 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:00.100369 (+ 59us) server_negotiation.cc:299] Negotiation successful
0810 23:03:00.100387 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260810 23:03:00.107404 13749 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:03:00.107875 13749 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:03:00.114151 13749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17f8eabf3c294ba0bc9667a781372a27: Generated new TSK 0
I20260810 23:03:00.114356 13749 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:03:00.116654 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13611
2026-08-10T23:03:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:03:00 dist-test-slave-rwrg krb5kdc[13793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:03:00 dist-test-slave-rwrg krb5kdc[13793](info): set up 2 sockets
Aug 10 23:03:00 dist-test-slave-rwrg krb5kdc[13793](info): commencing operation
krb5kdc: starting...
W20260810 23:03:01.385181 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:03:01 dist-test-slave-rwrg krb5kdc[13793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:03:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:03:01.509536 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34151
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:34151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:03:01.590801 13816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:01.591055 13816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:01.591094 13816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:01.592228 13816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:01.592299 13816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:01.592322 13816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:01.592342 13816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:01.592371 13816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:01.594158 13816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:34151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:34151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:01.594462 13816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:01.594717 13816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:01.599768 13822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:01.599763 13827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:01.599848 13816 server_base.cc:1034] running on GCE node
W20260810 23:03:01.599771 13821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:01.600296 13816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:01.600677 13816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:01.601850 13816 hybrid_clock.cc:648] HybridClock initialized: now 1786402981601846 us; error 38 us; skew 500 ppm
Aug 10 23:03:01 dist-test-slave-rwrg krb5kdc[13793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402981, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:01.607829 13816 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:03:01.608356 13816 webserver.cc:460] Webserver started at http://127.6.47.190:33621/ using document root <none> and password file <none>
I20260810 23:03:01.608649 13816 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:01.608719 13816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:01.608860 13816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:01.609961 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "f5a617c715f64fc5ae59089c52c94e11"
format_stamp: "Formatted at 2026-08-10 23:03:01 on dist-test-slave-rwrg"
server_key: "4fef0a61f77c005107f604b72c1dff57"
server_key_iv: "64e05f4e73a99320f893bb70bf493d83"
server_key_version: "encryptionkey@0"
I20260810 23:03:01.610348 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "f5a617c715f64fc5ae59089c52c94e11"
format_stamp: "Formatted at 2026-08-10 23:03:01 on dist-test-slave-rwrg"
server_key: "4fef0a61f77c005107f604b72c1dff57"
server_key_iv: "64e05f4e73a99320f893bb70bf493d83"
server_key_version: "encryptionkey@0"
I20260810 23:03:01.611785 13816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:03:01.612670 13835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:01.612859 13816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 23:03:01.612967 13816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "f5a617c715f64fc5ae59089c52c94e11"
format_stamp: "Formatted at 2026-08-10 23:03:01 on dist-test-slave-rwrg"
server_key: "4fef0a61f77c005107f604b72c1dff57"
server_key_iv: "64e05f4e73a99320f893bb70bf493d83"
server_key_version: "encryptionkey@0"
I20260810 23:03:01.613052 13816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:01.616405 13816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:01.618229 13816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:01.618378 13816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:01.622687 13816 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:34151
I20260810 23:03:01.622715 13939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:34151 every 8 connection(s)
I20260810 23:03:01.623030 13816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:03:01.623970 13940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:01.624627 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 13816
I20260810 23:03:01.624742 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:03:01.624917 6334 external_mini_cluster.cc:1428] Setting key 65c5204bdd562a7b2ddc2e9d0637d57d
I20260810 23:03:01.627213 13940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11: Bootstrap starting.
I20260810 23:03:01.627424 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:01.625511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60971 (local address 127.6.47.190:34151)
0810 23:03:01.625713 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:01.625722 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:01.625754 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:03:01.626138 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:01.626144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:01.626167 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:01.626279 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:01.627001 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:01.627005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:01.627005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:03:01.627009 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:01.627044 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:03:01.627053 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:01.627105 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:01.627191 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 23:03:01.627241 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260810 23:03:01.627898 13940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:01.628249 13940 log.cc:824] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:01.628399 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:03:01.629098 13940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11: No bootstrap required, opened a new log
I20260810 23:03:01.631011 13943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:01.630060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51527 (local address 127.6.47.190:34151)
0810 23:03:01.630100 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:01.630102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:01.630184 (+ 82us) server_negotiation.cc:407] Connection header received
0810 23:03:01.630239 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:01.630239 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:01.630243 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:01.630289 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:01.630808 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:01.630809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:01.630809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 23:03:01.630812 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:01.630823 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 23:03:01.630826 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:01.630869 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:01.630921 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 23:03:01.630931 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260810 23:03:01.631886 13940 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } }
I20260810 23:03:01.632021 13940 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 23:03:01.632045 13940 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a617c715f64fc5ae59089c52c94e11, State: Initialized, Role: FOLLOWER
I20260810 23:03:01.632130 13940 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } }
I20260810 23:03:01.632190 13940 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 23:03:01.632212 13940 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 23:03:01.632243 13940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:01.632822 13940 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } }
I20260810 23:03:01.632911 13940 leader_election.cc:302] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5a617c715f64fc5ae59089c52c94e11; no voters:
I20260810 23:03:01.633031 13940 leader_election.cc:288] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 23:03:01.633113 13951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:03:01.633279 13940 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:01.633288 13951 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [term 1 LEADER]: Becoming Leader. State: Replica: f5a617c715f64fc5ae59089c52c94e11, State: Running, Role: LEADER
I20260810 23:03:01.633442 13951 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } }
I20260810 23:03:01.633885 13954 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5a617c715f64fc5ae59089c52c94e11. Latest consensus state: current_term: 1 leader_uuid: "f5a617c715f64fc5ae59089c52c94e11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } } }
I20260810 23:03:01.633987 13954 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:01.634168 13953 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5a617c715f64fc5ae59089c52c94e11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5a617c715f64fc5ae59089c52c94e11" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 34151 } } }
I20260810 23:03:01.634227 13968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:03:01.634485 13953 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:01.634969 13968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:03:01.636598 13968 catalog_manager.cc:1347] Generated new cluster ID: e13e2d28380c43d69091ed5621000391
I20260810 23:03:01.636662 13968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:03:01.645805 13968 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:03:01.645982 13968 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:03:01.653828 13968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5a617c715f64fc5ae59089c52c94e11: Generated new TSK 0
I20260810 23:03:01.654021 13968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 23:03:01.658419 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 13816
2026-08-10T23:03:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1536 ms)
[----------] 18 tests from AuthTokenIssuingTest (28658 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:03:01 dist-test-slave-rwrg krb5kdc[13993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:03:01 dist-test-slave-rwrg krb5kdc[13993](info): set up 2 sockets
Aug 10 23:03:01 dist-test-slave-rwrg krb5kdc[13993](info): commencing operation
krb5kdc: starting...
W20260810 23:03:02.946105 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:03:03.071640 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38241
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:03:03.152608 14030 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:03.152812 14030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:03.152828 14030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:03.152848 14030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:03.153980 14030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:03.154037 14030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:03.154052 14030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:03.154064 14030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:03.154119 14030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:03.155570 14030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:38241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:03.155763 14030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:03.156006 14030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:03.160616 14047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:03.160627 14045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:03.160691 14052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:03.160939 14030 server_base.cc:1034] running on GCE node
I20260810 23:03:03.161186 14030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:03.161432 14030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:03.162572 14030 hybrid_clock.cc:648] HybridClock initialized: now 1786402983162559 us; error 31 us; skew 500 ppm
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:03.165060 14030 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:03:03.165474 14030 webserver.cc:460] Webserver started at http://127.6.47.190:38369/ using document root <none> and password file <none>
I20260810 23:03:03.165725 14030 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:03.165795 14030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:03.165949 14030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:03.166963 14030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "75f016c74d1442bdab345914583a8de8"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "ebd92d88fc7d2dfdd7586c6bbdb90495"
server_key_iv: "4a5446b21784a33d2d2cda3151eadaa1"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.167320 14030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "75f016c74d1442bdab345914583a8de8"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "ebd92d88fc7d2dfdd7586c6bbdb90495"
server_key_iv: "4a5446b21784a33d2d2cda3151eadaa1"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.168634 14030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:03:03.169361 14062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:03.169539 14030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:03:03.169659 14030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "75f016c74d1442bdab345914583a8de8"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "ebd92d88fc7d2dfdd7586c6bbdb90495"
server_key_iv: "4a5446b21784a33d2d2cda3151eadaa1"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.169735 14030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:03.178565 14030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:03.180405 14030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:03.180593 14030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:03.184572 14030 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:38241
I20260810 23:03:03.184587 14158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:38241 every 8 connection(s)
I20260810 23:03:03.184950 14030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:03:03.185981 14159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:03.186287 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14030
I20260810 23:03:03.186408 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:03:03.186574 6334 external_mini_cluster.cc:1428] Setting key c1f307a2d65707d7fd72464197932ebf
I20260810 23:03:03.187685 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.189@KRBTEST.COM; defaulting to no policy
I20260810 23:03:03.198401 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.188565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.6.47.190:38241)
0810 23:03:03.188699 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.188703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.188735 (+ 32us) server_negotiation.cc:407] Connection header received
0810 23:03:03.189303 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.189306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.189332 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.189429 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.189982 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.191266 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.192241 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.192818 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.192949 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.194919 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.194929 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.194930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.194938 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.196757 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.197101 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.197103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.197103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.197255 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.197506 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.197507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.197507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.197717 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.197808 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.198000 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.198241 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260810 23:03:03.198401 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.188403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38241 (local address 127.0.0.1:39024)
0810 23:03:03.188605 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.188649 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.189130 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.189434 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.189437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.189719 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.189877 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.189881 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.191369 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.191370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.192144 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.192147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.192927 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.192928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.192984 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.193513 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.193521 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.194828 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.196829 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.196835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.196837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.197024 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.197346 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.197347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.197347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.197433 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.197788 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.197793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.197854 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.197964 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.198158 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.189@KRBTEST.COM" created.
I20260810 23:03:03.199265 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:03.199637 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.199508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37516)
0810 23:03:03.199559 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.199614 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 23:03:03.199759 14068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
I20260810 23:03:03.200187 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.200077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37524)
0810 23:03:03.200140 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.200164 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 23:03:03.200289 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:34719: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
I20260810 23:03:03.214740 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:34719
--webserver_interface=127.6.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:03:03.249553 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } attempt: 1
I20260810 23:03:03.250126 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.249960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37538)
0810 23:03:03.250039 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.250088 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 23:03:03.250528 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.250425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37544)
0810 23:03:03.250472 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.250500 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:03:03.250659 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:34719: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
W20260810 23:03:03.297236 14171 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:03.297451 14171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:03.297468 14171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:03.297488 14171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:03.298631 14171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:03.298673 14171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:03.298686 14171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:03.298725 14171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:03.298758 14171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:03.300257 14171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:03.300575 14171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:03.300815 14171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:03.306145 14180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:03.306200 14178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:03.306214 14171 server_base.cc:1034] running on GCE node
W20260810 23:03:03.306208 14182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:03.306517 14171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:03.306754 14171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:03.307906 14171 hybrid_clock.cc:648] HybridClock initialized: now 1786402983307876 us; error 48 us; skew 500 ppm
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:03.310359 14171 init.cc:377] Logged in from keytab as kudu/127.6.47.189@KRBTEST.COM (short username kudu)
I20260810 23:03:03.310858 14171 webserver.cc:460] Webserver started at http://127.6.47.189:39513/ using document root <none> and password file <none>
I20260810 23:03:03.311095 14171 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:03.311165 14171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:03.311317 14171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:03.312364 14171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data/instance:
uuid: "0336603df4e4448aa76b30d6b1858f9b"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "bff687dbf122907c461164c0b3ceb965"
server_key_iv: "82acd13bd9e55a0ca347b0be8a5b2051"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.312758 14171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal/instance:
uuid: "0336603df4e4448aa76b30d6b1858f9b"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "bff687dbf122907c461164c0b3ceb965"
server_key_iv: "82acd13bd9e55a0ca347b0be8a5b2051"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.314059 14171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:03:03.314815 14190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:03.314985 14171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:03:03.315090 14171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "0336603df4e4448aa76b30d6b1858f9b"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "bff687dbf122907c461164c0b3ceb965"
server_key_iv: "82acd13bd9e55a0ca347b0be8a5b2051"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.315173 14171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:03.318851 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } attempt: 2
I20260810 23:03:03.319244 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.319105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37548)
0810 23:03:03.319166 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.319208 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 23:03:03.319613 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.319516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37558)
0810 23:03:03.319562 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.319590 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 23:03:03.319782 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:34719: Network error: Client connection negotiation failed: client connection to 127.6.47.189:34719: connect: Connection refused (error 111)
I20260810 23:03:03.327219 14171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:03.329187 14171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:03.329371 14171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:03.333556 14171 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.189:34719
I20260810 23:03:03.333611 14283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.189:34719 every 8 connection(s)
I20260810 23:03:03.333998 14171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
I20260810 23:03:03.334977 14284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:03.336584 14284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:03.339618 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14171
I20260810 23:03:03.339715 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-1/wal/instance
I20260810 23:03:03.339883 6334 external_mini_cluster.cc:1428] Setting key 95dcadf1db08ba566c3b4eea99e4934f
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:03.345769 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.337275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.6.47.190:38241)
0810 23:03:03.337318 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.337321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.337489 (+ 168us) server_negotiation.cc:407] Connection header received
0810 23:03:03.337935 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.337937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.337947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.338002 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.338615 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.339080 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.340214 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.340560 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.340647 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.342886 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.342887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.342888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.342896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.344267 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.344756 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.344757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.344757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.344821 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.345094 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.345095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.345095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.345216 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.345288 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.345580 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.345664 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:03.345940 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.337226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38241 (local address 127.0.0.1:39036)
0810 23:03:03.337381 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.337422 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.337852 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.338102 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.338105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.338345 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.338536 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.338540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.339185 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.339186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.340118 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.340121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.340665 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.340666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.340774 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.341282 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.341291 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.342774 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.344359 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.344368 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.344370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.344678 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.344894 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.344895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.344895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.345015 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.345293 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.345302 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.345441 (+ 139us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.345546 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.345707 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260810 23:03:03.346573 14284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.188@KRBTEST.COM" created.
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:03.354933 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.346925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.6.47.189:34719)
0810 23:03:03.347108 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.347112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.347128 (+ 16us) server_negotiation.cc:407] Connection header received
0810 23:03:03.347144 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.347145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.347154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.347230 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.347517 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.348206 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.348984 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.349309 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.349405 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.351934 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.351944 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.351944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.351953 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.353625 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.354037 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.354040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.354040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.354093 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.354306 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.354307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.354307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.354437 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.354498 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.354690 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.354830 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260810 23:03:03.354975 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.346839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37570)
0810 23:03:03.346888 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.346899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.346936 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.347248 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.347250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.347387 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.347433 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.347435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.348299 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.348300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.348899 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.348901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.349404 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.349405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.349475 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.349913 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.349919 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.351846 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.353713 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.353715 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.353715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.353949 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.354165 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.354166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.354166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.354239 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.354506 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.354507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.354559 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.354651 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.354768 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:03:03.355711 14284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:03.356139 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.355987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33578)
0810 23:03:03.356047 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.356105 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 23:03:03.356295 14199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
I20260810 23:03:03.356835 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.356720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33582)
0810 23:03:03.356772 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.356806 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:03:03.356958 14284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:37251: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
I20260810 23:03:03.368075 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:37251
--webserver_interface=127.6.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:34307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:03:03.406234 14284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } attempt: 1
I20260810 23:03:03.406862 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.406685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33592)
0810 23:03:03.406766 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.406820 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 23:03:03.407250 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.407147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33604)
0810 23:03:03.407199 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.407229 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 23:03:03.407384 14284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:37251: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
I20260810 23:03:03.410916 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } attempt: 3
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:03.418079 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.411234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37572)
0810 23:03:03.411298 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.411309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.411373 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.411534 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.411535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.411714 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.411787 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.411788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.412535 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.412536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.413074 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.413076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.413481 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.413482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.413584 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.414056 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.414061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.415263 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.416942 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.416943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.416944 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.417116 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.417351 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.417352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.417352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.417439 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.417735 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.417736 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.417786 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.417869 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.417976 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 23:03:03.418079 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.411340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.6.47.189:34719)
0810 23:03:03.411404 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.411407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.411416 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:03:03.411444 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.411445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.411463 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.411526 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.411871 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.412401 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.413170 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.413400 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.413482 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.415357 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.415358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.415359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.415366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.416826 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.417213 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.417215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.417215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.417275 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.417492 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.417493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.417493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.417644 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.417745 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.417909 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.417985 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 23:03:03.418460 14159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:03.418848 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.418760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33620)
0810 23:03:03.418794 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.418825 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260810 23:03:03.419201 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.419139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33630)
0810 23:03:03.419162 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.419188 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260810 23:03:03.419288 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:37251: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
I20260810 23:03:03.450421 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } attempt: 1
I20260810 23:03:03.450865 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.450730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33646)
0810 23:03:03.450797 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.450835 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 23:03:03.451272 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.451118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33654)
0810 23:03:03.451189 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.451236 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 23:03:03.451406 14159 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:37251: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
W20260810 23:03:03.454635 14303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:03.454840 14303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:03.454857 14303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:03.454876 14303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:03.455938 14303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:03.455974 14303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:03.455988 14303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:03.456001 14303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:03.456013 14303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:03.457507 14303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:34307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:38241,127.6.47.189:34719,127.6.47.188:37251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:03.457722 14303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:03.457894 14303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:03.462692 14311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:03.462852 14303 server_base.cc:1034] running on GCE node
W20260810 23:03:03.462680 14313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:03.462729 14315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:03.463177 14303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:03.463501 14303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:03.464670 14303 hybrid_clock.cc:648] HybridClock initialized: now 1786402983464656 us; error 43 us; skew 500 ppm
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:03.467063 14303 init.cc:377] Logged in from keytab as kudu/127.6.47.188@KRBTEST.COM (short username kudu)
I20260810 23:03:03.467486 14303 webserver.cc:460] Webserver started at http://127.6.47.188:36969/ using document root <none> and password file <none>
I20260810 23:03:03.467767 14303 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:03.467839 14303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:03.467985 14303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:03.469095 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data/instance:
uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "5ef9df8d3e2f73602777533afc0a1f8c"
server_key_iv: "66784dbf1dbf049b917ea70ff0546f3d"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.469482 14303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal/instance:
uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "5ef9df8d3e2f73602777533afc0a1f8c"
server_key_iv: "66784dbf1dbf049b917ea70ff0546f3d"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.470855 14303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:03:03.471555 14330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:03.471726 14303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:03:03.471832 14303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f"
format_stamp: "Formatted at 2026-08-10 23:03:03 on dist-test-slave-rwrg"
server_key: "5ef9df8d3e2f73602777533afc0a1f8c"
server_key_iv: "66784dbf1dbf049b917ea70ff0546f3d"
server_key_version: "encryptionkey@0"
I20260810 23:03:03.471931 14303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:03.475587 14284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } attempt: 2
I20260810 23:03:03.475968 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.475850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33656)
0810 23:03:03.475907 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.475943 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 23:03:03.476326 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.476235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33672)
0810 23:03:03.476280 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.476303 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 23:03:03.476414 14284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:37251: Network error: Client connection negotiation failed: client connection to 127.6.47.188:37251: connect: Connection refused (error 111)
I20260810 23:03:03.488284 14303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:03.490288 14303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:03.490456 14303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:03.494504 14303 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.188:37251
I20260810 23:03:03.494539 14424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.188:37251 every 8 connection(s)
I20260810 23:03:03.494912 14303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
I20260810 23:03:03.495949 14425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:03.497613 14425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:03.503706 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14303
I20260810 23:03:03.503820 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786402701710467-6334-0/minicluster-data/master-2/wal/instance
I20260810 23:03:03.504005 6334 external_mini_cluster.cc:1428] Setting key 74d3f5a71405594a0d5d7910d62035a6
I20260810 23:03:03.506851 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.498331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.6.47.190:38241)
0810 23:03:03.498416 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.498419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.498598 (+ 179us) server_negotiation.cc:407] Connection header received
0810 23:03:03.499089 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.499091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.499104 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.499157 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.499767 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.500243 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.501302 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.501511 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.501589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.503704 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.503705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.503706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.503713 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.505459 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.505925 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.505926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.505926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.505994 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.506229 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.506229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.506230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.506362 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.506437 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.506720 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.506788 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:03:03.507069 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.498290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:38241 (local address 127.0.0.1:39040)
0810 23:03:03.498482 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.498517 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.498976 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.499225 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.499229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.499459 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.499640 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.499645 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.500341 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.500342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.501204 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.501207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.501611 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.501612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.501740 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.502255 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.502263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.503603 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.505580 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.505587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.505589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.505848 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.506065 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.506066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.506066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.506163 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.506434 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.506438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.506578 (+ 140us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.506700 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.506854 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260810 23:03:03.507691 14425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:03.513840 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.504462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.6.47.188:37251)
0810 23:03:03.504636 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.504641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.504756 (+ 115us) server_negotiation.cc:407] Connection header received
0810 23:03:03.504838 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.504841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.504855 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.504957 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.505549 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.506475 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.507502 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.507871 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.507926 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.510638 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.510647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.510647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.510654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.512466 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.512896 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.512899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.512900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.512966 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.513206 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.513207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.513207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.513403 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.513555 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.513625 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.513707 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260810 23:03:03.515079 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.507960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:34719 (local address 127.0.0.1:37576)
0810 23:03:03.508028 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.508039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.508077 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.508193 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.508194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.508341 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.508395 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.508397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.509114 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.509116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.509775 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.509778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.510266 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.510267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.510347 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.510740 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.510744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.512095 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.513972 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.513972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.513973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.514177 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.514395 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.514395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.514396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.514459 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.514769 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.514770 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.514811 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.514878 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.514975 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 23:03:03.515107 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.508049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.6.47.189:34719)
0810 23:03:03.508101 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.508103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.508112 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:03:03.508135 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.508136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.508143 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.508186 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.508521 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.509018 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.509867 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.510195 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.510250 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.512189 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.512191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.512191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.512200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.513879 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.514253 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.514254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.514254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.514310 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.514528 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.514529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.514529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.514698 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.514836 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.514914 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.514998 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:03:03.515465 14425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:03.522583 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.515715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33690)
0810 23:03:03.515754 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.515762 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.515791 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.515907 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.515907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.516009 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.516035 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.516036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.516778 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.516779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.517435 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.517437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.517876 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.517877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.517923 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.518288 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.518292 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.519777 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.521471 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.521472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.521472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.521688 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.521918 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.521918 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.521919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.521981 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.522237 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.522238 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.522280 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.522358 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.522450 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 23:03:03.522688 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.514792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.6.47.190:38241)
0810 23:03:03.514831 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.514832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.514929 (+ 97us) server_negotiation.cc:407] Connection header received
0810 23:03:03.514973 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.514974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.514985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.515033 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.515701 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.516399 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.517308 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.517505 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.517571 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.519956 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.519958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.519958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.519965 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.521552 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.521918 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.521919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.521919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.521973 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.522172 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.522172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.522172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.522311 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.522469 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.522512 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.522585 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:03:03.522696 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.515769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.6.47.188:37251)
0810 23:03:03.515804 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.515806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.515812 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:03:03.515834 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.515835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.515843 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.515904 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.516099 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.516709 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.517565 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.517805 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.517877 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.519861 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.519862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.519863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.519868 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.521385 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.521772 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.521772 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.521773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.521829 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.522046 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.522047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.522047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.522171 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.522228 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.522403 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.522513 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":20}
I20260810 23:03:03.524840 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: Bootstrap starting.
I20260810 23:03:03.525908 14425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:03.526268 14425 log.cc:824] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:03.526566 14159 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } attempt: 2
I20260810 23:03:03.527194 14425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: No bootstrap required, opened a new log
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:03.529521 14425 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.529685 14425 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c32d102fdd4a7ab2cc81e2b0ba965f, State: Initialized, Role: FOLLOWER
I20260810 23:03:03.529845 14425 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.530202 14425 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:03.530175 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.530418 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: This master's current role is: FOLLOWER
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
I20260810 23:03:03.532105 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.523480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.6.47.189:34719)
0810 23:03:03.523540 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.523541 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.523549 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:03:03.523586 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.523587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.523597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.523646 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.524200 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.524718 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.525606 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.525816 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.525869 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.528446 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.528447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.528447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.528454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.530484 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.530901 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.530902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.530903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.530964 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.531198 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.531200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.531200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.531340 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.531520 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.531909 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.531982 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 23:03:03.532341 14452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:03:03.532415 14452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:03:03.534183 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:03:03.534483 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.526835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33702)
0810 23:03:03.526881 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.526890 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.526930 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.527158 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.527159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.527341 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.527386 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.527387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.527988 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.527989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.528608 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.528612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.529077 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.529079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.529146 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.529546 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.529550 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.530968 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.533260 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.533261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.533261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.533503 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.533810 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.533810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.533811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.533872 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.534188 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.534189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.534230 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.534295 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.534389 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 23:03:03.534821 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.526983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.6.47.188:37251)
0810 23:03:03.527037 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.527038 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.527045 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:03:03.527063 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.527064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.527072 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.527139 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.527460 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.527897 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.528713 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.528996 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.529059 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.531057 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.531059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.531059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.531066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.533172 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.533584 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.533585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.533586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.533645 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.533943 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.533944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.533944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.534092 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.534150 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.534649 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.534737 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 23:03:03.536427 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8: Bootstrap starting.
I20260810 23:03:03.535712 14111 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "75f016c74d1442bdab345914583a8de8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39054
I20260810 23:03:03.533771 14303 master_runner.cc:186] Error getting master registration for 127.6.47.190:38241: OK, instance_id { permanent_uuid: "75f016c74d1442bdab345914583a8de8" instance_seqno: 1786402983183447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:03:03.537047 14303 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:03:03.537314 14159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:03.537860 14159 log.cc:824] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:03.538733 14159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8: No bootstrap required, opened a new log
I20260810 23:03:03.540096 14159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.540241 14159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f016c74d1442bdab345914583a8de8, State: Initialized, Role: FOLLOWER
I20260810 23:03:03.540328 14159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.540705 14159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:03.540657 14458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.540751 14111 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 23:03:03.540774 14458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.540804 14111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:03.541697 14111 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.541975 14111 leader_election.cc:288] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [CANDIDATE]: Term 1 election: Requested vote from peers 0336603df4e4448aa76b30d6b1858f9b (127.6.47.189:34719), c2c32d102fdd4a7ab2cc81e2b0ba965f (127.6.47.188:37251)
I20260810 23:03:03.542470 14387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f016c74d1442bdab345914583a8de8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f"
I20260810 23:03:03.542587 14387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:03.543668 14387 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f016c74d1442bdab345914583a8de8 in term 1.
I20260810 23:03:03.543903 14071 leader_election.cc:302] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f016c74d1442bdab345914583a8de8, c2c32d102fdd4a7ab2cc81e2b0ba965f; no voters:
I20260810 23:03:03.544040 14458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:03:03.544196 14458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [term 1 LEADER]: Becoming Leader. State: Replica: 75f016c74d1442bdab345914583a8de8, State: Running, Role: LEADER
I20260810 23:03:03.544303 14458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.544771 14458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f016c74d1442bdab345914583a8de8. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.544852 14458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:03.546319 14030 master_runner.cc:186] Error getting master registration for 127.6.47.189:34719: OK, instance_id { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" instance_seqno: 1786402983332338 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:03:03.549296 14030 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:03:03.549445 14500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:03:03.549980 14500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:03:03.552807 14387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [term 1 FOLLOWER]: Refusing update from remote peer 75f016c74d1442bdab345914583a8de8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:03:03.553258 14458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:03:03.554628 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f016c74d1442bdab345914583a8de8. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.554738 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.554857 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.543578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.6.47.188:37251)
0810 23:03:03.543615 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.543617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.543835 (+ 218us) server_negotiation.cc:407] Connection header received
0810 23:03:03.543926 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.543927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.543936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.543965 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.544867 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.545632 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.546715 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.547031 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.547071 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.549868 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.549869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.549870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.549877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.551570 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.551967 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.551968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.551969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.552023 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.552220 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.552221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.552221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.552342 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.552409 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.552663 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.552745 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:03.556015 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.543246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.6.47.189:34719)
0810 23:03:03.543290 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.543293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.543583 (+ 290us) server_negotiation.cc:407] Connection header received
0810 23:03:03.543854 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.543855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.543868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.543901 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.545632 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.546326 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.547904 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.548193 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.548239 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.553076 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.553077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.553078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.553086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.554734 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.555140 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.555141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.555142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.555200 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.555408 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.555408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.555409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.555531 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.555613 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.555828 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.555915 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:03.557502 14458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.557605 14458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:03.558061 14434 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.558153 14434 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.563421 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.555620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.6.47.190:38241)
0810 23:03:03.555661 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.555663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.555747 (+ 84us) server_negotiation.cc:407] Connection header received
0810 23:03:03.555857 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.555858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.555869 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.555923 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.556543 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.557247 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.558259 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.558544 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.558589 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.559997 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.559998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.559999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.560005 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.561687 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.562282 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.562283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.562283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.562343 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.562628 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.562629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.562629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.562761 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.562863 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.563133 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.563274 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:03.564015 14500 catalog_manager.cc:1347] Generated new cluster ID: d99e1f3d3d9b4345b711ff75effa9e3f
I20260810 23:03:03.564098 14500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 23:03:03.567528 14284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } attempt: 3
Aug 10 23:03:03 dist-test-slave-rwrg krb5kdc[13993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402983, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
I20260810 23:03:03.575311 14288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.567863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:37251 (local address 127.0.0.1:33720)
0810 23:03:03.567904 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:03.567916 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:03.567959 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:03.568111 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:03.568112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:03.568280 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:03.568330 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.568332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.569115 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.569116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.569912 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:03.569915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.570359 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.570360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:03.570421 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.570888 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:03.570893 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:03.572357 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:03.574315 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.574316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.574316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.574520 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.574698 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:03.574699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:03.574699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:03.574756 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:03.575011 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:03.575012 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:03.575063 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:03:03.575121 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 23:03:03.575210 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260810 23:03:03.575644 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:03.567964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.6.47.188:37251)
0810 23:03:03.568001 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.568003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.568010 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:03:03.568031 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.568031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.568056 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.568105 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.568411 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.569031 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.570001 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:03.570288 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:03.570359 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:03.572446 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:03.572447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:03.572447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:03.572454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:03.574237 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.574593 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.574595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.574595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.574648 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:03.574814 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:03.574814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:03.574815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:03.574931 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:03.574986 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:03.575495 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 23:03:03.575564 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:03.577500 14284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: Bootstrap starting.
I20260810 23:03:03.578229 14284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:03.578536 14284 log.cc:824] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:03.585336 14284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: No bootstrap required, opened a new log
I20260810 23:03:03.587419 14284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.587561 14284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0336603df4e4448aa76b30d6b1858f9b, State: Initialized, Role: FOLLOWER
I20260810 23:03:03.587636 14284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } }
I20260810 23:03:03.587958 14284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:03.588441 14231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "75f016c74d1442bdab345914583a8de8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0336603df4e4448aa76b30d6b1858f9b"
I20260810 23:03:03.588575 14231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:03.589450 14231 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f016c74d1442bdab345914583a8de8 in term 1.
I20260810 23:03:03.589928 14230 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [term 1 FOLLOWER]: Refusing update from remote peer 75f016c74d1442bdab345914583a8de8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:03:03.590294 14498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 23:03:03.595950 14560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:03:03.596242 14560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:03:03.596141 14526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.596449 14526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.596578 14557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75f016c74d1442bdab345914583a8de8. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.596689 14557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.596598 14526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.596786 14526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:03.597874 14555 mvcc.cc:205] Tried to move back new op lower bound from 7317106620627423232 to 7317106620598452224. Current Snapshot: MvccSnapshot[applied={T|T < 7317106620627423232}]
I20260810 23:03:03.600759 14498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "75f016c74d1442bdab345914583a8de8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75f016c74d1442bdab345914583a8de8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 38241 } } peers { permanent_uuid: "0336603df4e4448aa76b30d6b1858f9b" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 34719 } } peers { permanent_uuid: "c2c32d102fdd4a7ab2cc81e2b0ba965f" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 37251 } } }
I20260810 23:03:03.600867 14498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:03.602999 14500 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:03:03.603724 14500 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:03:03.611618 14500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75f016c74d1442bdab345914583a8de8: Generated new TSK 0
I20260810 23:03:03.611779 14500 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:03:03.642094 14432 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.640435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.6.47.188:37251)
0810 23:03:03.640511 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.640514 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.641418 (+ 904us) server_negotiation.cc:407] Connection header received
0810 23:03:03.641530 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.641531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.641542 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.641573 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.642024 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33736: BlockingRecv error: recv got EOF from 127.0.0.1:33736 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 23:03:03.642889 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.640871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.6.47.189:34719)
0810 23:03:03.640915 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.640917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.642355 (+ 1438us) server_negotiation.cc:407] Connection header received
0810 23:03:03.642457 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.642458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.642469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.642499 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.642825 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37604: BlockingRecv error: recv got EOF from 127.0.0.1:37604 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260810 23:03:03.643663 14168 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.641143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.6.47.190:38241)
0810 23:03:03.641203 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.641205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.643112 (+ 1907us) server_negotiation.cc:407] Connection header received
0810 23:03:03.643146 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.643147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.643159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.643231 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.643597 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39076: BlockingRecv error: recv got EOF from 127.0.0.1:39076 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260810 23:03:03.645702 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.644967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.6.47.189:34719)
0810 23:03:03.645011 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.645012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.645236 (+ 224us) server_negotiation.cc:407] Connection header received
0810 23:03:03.645334 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.645335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.645353 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.645380 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.645658 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37606: BlockingRecv error: recv got EOF from 127.0.0.1:37606 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:03:03.648777 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.647905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.6.47.189:34719)
0810 23:03:03.647937 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.647938 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.648155 (+ 217us) server_negotiation.cc:407] Connection header received
0810 23:03:03.648269 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.648270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.648280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.648308 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.648735 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37612: BlockingRecv error: recv got EOF from 127.0.0.1:37612 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:03:03.652855 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.651934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.6.47.189:34719)
0810 23:03:03.651979 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.651981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.652216 (+ 235us) server_negotiation.cc:407] Connection header received
0810 23:03:03.652317 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.652318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.652329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.652359 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.652810 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37626: BlockingRecv error: recv got EOF from 127.0.0.1:37626 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 23:03:03.658828 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.657972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.6.47.189:34719)
0810 23:03:03.658006 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.658007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.658271 (+ 264us) server_negotiation.cc:407] Connection header received
0810 23:03:03.658373 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.658374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.658385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.658414 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.658780 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37636: BlockingRecv error: recv got EOF from 127.0.0.1:37636 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:03:03.669229 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.668221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.6.47.189:34719)
0810 23:03:03.668265 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.668267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.668602 (+ 335us) server_negotiation.cc:407] Connection header received
0810 23:03:03.668732 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.668733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.668746 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.668777 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.669169 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37638: BlockingRecv error: recv got EOF from 127.0.0.1:37638 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:03:03.687704 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.686730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.6.47.189:34719)
0810 23:03:03.686775 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.686778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.687076 (+ 298us) server_negotiation.cc:407] Connection header received
0810 23:03:03.687186 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.687188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.687201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.687231 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.687645 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37648: BlockingRecv error: recv got EOF from 127.0.0.1:37648 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260810 23:03:03.721845 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.720993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.6.47.189:34719)
0810 23:03:03.721043 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.721045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.721210 (+ 165us) server_negotiation.cc:407] Connection header received
0810 23:03:03.721294 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.721294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.721306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.721380 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.721781 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37658: BlockingRecv error: recv got EOF from 127.0.0.1:37658 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:03:03.787798 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.787011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.6.47.189:34719)
0810 23:03:03.787088 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.787090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.787173 (+ 83us) server_negotiation.cc:407] Connection header received
0810 23:03:03.787246 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.787247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.787258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.787305 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.787713 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37670: BlockingRecv error: recv got EOF from 127.0.0.1:37670 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 23:03:03.917909 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:03.917067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.6.47.189:34719)
0810 23:03:03.917157 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:03.917159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:03.917278 (+ 119us) server_negotiation.cc:407] Connection header received
0810 23:03:03.917384 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:03.917385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:03.917398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:03.917473 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:03.917839 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37674: BlockingRecv error: recv got EOF from 127.0.0.1:37674 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 23:03:04.175889 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:04.175229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37686 (local address 127.6.47.189:34719)
0810 23:03:04.175310 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.175312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.175323 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:03:04.175408 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.175409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.175419 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.175470 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:04.175838 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37686: BlockingRecv error: recv got EOF from 127.0.0.1:37686 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 23:03:04.532928 14452 catalog_manager.cc:1259] Loaded cluster ID: d99e1f3d3d9b4345b711ff75effa9e3f
I20260810 23:03:04.533022 14452 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: loading cluster ID for follower catalog manager: success
I20260810 23:03:04.533812 14452 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: acquiring CA information for follower catalog manager: success
I20260810 23:03:04.534123 14452 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2c32d102fdd4a7ab2cc81e2b0ba965f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:03:04.596807 14560 catalog_manager.cc:1259] Loaded cluster ID: d99e1f3d3d9b4345b711ff75effa9e3f
I20260810 23:03:04.596890 14560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: loading cluster ID for follower catalog manager: success
I20260810 23:03:04.597711 14560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: acquiring CA information for follower catalog manager: success
I20260810 23:03:04.598032 14560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0336603df4e4448aa76b30d6b1858f9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:03:04.691737 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:04.689118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.6.47.189:34719)
0810 23:03:04.689275 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.689278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.689290 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:03:04.689369 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.689370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.689424 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.689501 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:04.689652 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.690149 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.690946 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.691150 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.691205 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:04.691357 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:04.691477 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:04.691598 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:04.691650 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 23:03:04.691669 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260810 23:03:04.695904 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:04.693629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.6.47.188:37251)
0810 23:03:04.693746 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.693748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.693757 (+ 9us) server_negotiation.cc:407] Connection header received
0810 23:03:04.693806 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.693806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.693842 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.693910 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:04.694019 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.694504 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.695168 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.695361 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.695409 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:04.695544 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:04.695644 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:04.695699 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:04.695760 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 23:03:04.695776 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260810 23:03:04.700711 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:04.697679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.6.47.188:37251)
0810 23:03:04.697746 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.697747 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.698018 (+ 271us) server_negotiation.cc:407] Connection header received
0810 23:03:04.698105 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.698107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.698117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.698169 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:04.698329 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.698757 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.699868 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.700157 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.700249 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:04.700384 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:04.700416 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:04.700469 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:04.700583 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 23:03:04.700614 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 23:03:04.700785 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:04.697524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56434 (local address 127.6.47.189:34719)
0810 23:03:04.697705 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.697707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.697925 (+ 218us) server_negotiation.cc:407] Connection header received
0810 23:03:04.697955 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.697956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.697965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.698013 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:04.698228 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.698910 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.699976 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.700255 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.700316 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:04.700510 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:04.700545 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:04.700593 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:04.700688 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 23:03:04.700709 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260810 23:03:04.700861 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:04.697578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.6.47.190:38241)
0810 23:03:04.697738 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:04.697740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:04.697804 (+ 64us) server_negotiation.cc:407] Connection header received
0810 23:03:04.697868 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:04.697869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:04.697923 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:04.698015 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:04.698161 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.698931 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.699731 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:04.700048 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:04.700114 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:04.700333 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:04.700440 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:04.700531 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:04.700763 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 23:03:04.700788 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260810 23:03:04.702901 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14030
I20260810 23:03:04.721696 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14171
I20260810 23:03:04.733649 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14303
2026-08-10T23:03:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3067 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 23:03:04 dist-test-slave-rwrg krb5kdc[14720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 23:03:04 dist-test-slave-rwrg krb5kdc[14720](info): set up 2 sockets
Aug 10 23:03:04 dist-test-slave-rwrg krb5kdc[14720](info): commencing operation
krb5kdc: starting...
W20260810 23:03:06.034590 6334 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/test-user.keytab.
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T23:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T23:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
Entry for principal kudu/127.6.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab.
I20260810 23:03:06.157723 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41369
--webserver_interface=127.6.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 23:03:06.239758 14743 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:06.240036 14743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:06.240067 14743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:06.240109 14743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:06.241247 14743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:06.241341 14743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:06.241367 14743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:06.241393 14743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:06.241423 14743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:06.242874 14743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.190.keytab
--principal=kudu/127.6.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.190:41369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:06.243162 14743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:06.243409 14743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:06.248587 14761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:06.248637 14755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:06.248555 14757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:06.248752 14743 server_base.cc:1034] running on GCE node
I20260810 23:03:06.249086 14743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:06.249411 14743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:06.250591 14743 hybrid_clock.cc:648] HybridClock initialized: now 1786402986250577 us; error 35 us; skew 500 ppm
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:06.253111 14743 init.cc:377] Logged in from keytab as kudu/127.6.47.190@KRBTEST.COM (short username kudu)
I20260810 23:03:06.253486 14743 webserver.cc:460] Webserver started at http://127.6.47.190:43727/ using document root <none> and password file <none>
I20260810 23:03:06.253729 14743 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:06.253805 14743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:06.253957 14743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:06.254962 14743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data/instance:
uuid: "7602705a2d3f4ab8b089a5b2458a01d8"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "cd3f1ae4c11608cffb193224697c1282"
server_key_iv: "bd05a20a40be7c9ecd0a174e956120bf"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.255357 14743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance:
uuid: "7602705a2d3f4ab8b089a5b2458a01d8"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "cd3f1ae4c11608cffb193224697c1282"
server_key_iv: "bd05a20a40be7c9ecd0a174e956120bf"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.256701 14743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260810 23:03:06.257452 14771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:06.257634 14743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260810 23:03:06.257745 14743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal
uuid: "7602705a2d3f4ab8b089a5b2458a01d8"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "cd3f1ae4c11608cffb193224697c1282"
server_key_iv: "bd05a20a40be7c9ecd0a174e956120bf"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.257822 14743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:06.267437 14743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:06.269287 14743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:06.269454 14743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:06.273625 14743 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.190:41369
I20260810 23:03:06.273663 14874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.190:41369 every 8 connection(s)
I20260810 23:03:06.274039 14743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/data/info.pb
I20260810 23:03:06.275056 14876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:06.276640 14876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:06.282917 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14743
I20260810 23:03:06.283036 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-0/wal/instance
I20260810 23:03:06.283244 6334 external_mini_cluster.cc:1428] Setting key e71530ceeb3c22e5d133180e435638a8
I20260810 23:03:06.286193 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.277347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.6.47.190:41369)
0810 23:03:06.277529 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.277533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.277561 (+ 28us) server_negotiation.cc:407] Connection header received
0810 23:03:06.278041 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.278043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.278058 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.278140 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.278610 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.279384 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.280240 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.280614 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.280698 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.282853 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.282861 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.282862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.282868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.284474 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.284937 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.284940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.284940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.285040 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.285284 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.285285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.285285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.285499 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.285592 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.285810 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.285950 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260810 23:03:06.286202 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.277277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41369 (local address 127.0.0.1:37616)
0810 23:03:06.277430 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.277478 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.277910 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.278146 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.278148 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.278376 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.278521 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.278525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.279474 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.279475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.280156 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.280158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.280724 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.280725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.280781 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.281379 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.281387 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.282758 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.284626 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.284634 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.284636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.284830 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.285123 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.285123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.285124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.285211 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.285583 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.285589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.285652 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.285757 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.285999 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":106,"thread_start_us":68,"threads_started":1}
I20260810 23:03:06.286952 14876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:06.287305 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.287191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60486)
0810 23:03:06.287241 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.287284 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:03:06.287472 14778 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
I20260810 23:03:06.287914 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.287823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60496)
0810 23:03:06.287872 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.287896 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 23:03:06.288067 14876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:36405: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.6.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
Entry for principal kudu/127.6.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab.
I20260810 23:03:06.311457 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:36405
--webserver_interface=127.6.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:03:06.337333 14876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } attempt: 1
I20260810 23:03:06.337873 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.337705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60506)
0810 23:03:06.337780 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.337831 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 23:03:06.338294 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.338186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60518)
0810 23:03:06.338239 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.338269 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 23:03:06.338398 14876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:36405: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
W20260810 23:03:06.396929 14892 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:06.397138 14892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:06.397156 14892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:06.397174 14892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:06.398226 14892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:06.398257 14892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:06.398272 14892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:06.398284 14892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:06.398295 14892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:06.399674 14892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.189.keytab
--principal=kudu/127.6.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.189:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:06.399869 14892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:06.400053 14892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:06.405019 14908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:06.405041 14892 server_base.cc:1034] running on GCE node
W20260810 23:03:06.405004 14912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:06.405007 14906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:06.405537 14892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:06.405864 14892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:06.406529 14876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } attempt: 2
I20260810 23:03:06.406926 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.406808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60524)
0810 23:03:06.406867 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.406901 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 23:03:06.407047 14892 hybrid_clock.cc:648] HybridClock initialized: now 1786402986407016 us; error 46 us; skew 500 ppm
I20260810 23:03:06.407302 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.407173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60530)
0810 23:03:06.407242 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.407276 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 23:03:06.407407 14876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.189:36405: Network error: Client connection negotiation failed: client connection to 127.6.47.189:36405: connect: Connection refused (error 111)
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:06.409713 14892 init.cc:377] Logged in from keytab as kudu/127.6.47.189@KRBTEST.COM (short username kudu)
I20260810 23:03:06.410096 14892 webserver.cc:460] Webserver started at http://127.6.47.189:33245/ using document root <none> and password file <none>
I20260810 23:03:06.410295 14892 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:06.410336 14892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:06.410423 14892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:06.411424 14892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data/instance:
uuid: "5d2a786f26674b5db91273eb60fafb7a"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "8be4e4b7c545277c8656ee884538b120"
server_key_iv: "7a6a1d4d089e927d7c2b5b045a0b1a94"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.411720 14892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal/instance:
uuid: "5d2a786f26674b5db91273eb60fafb7a"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "8be4e4b7c545277c8656ee884538b120"
server_key_iv: "7a6a1d4d089e927d7c2b5b045a0b1a94"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.413268 14892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 23:03:06.413973 14927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:06.414188 14892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:03:06.414273 14892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal
uuid: "5d2a786f26674b5db91273eb60fafb7a"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "8be4e4b7c545277c8656ee884538b120"
server_key_iv: "7a6a1d4d089e927d7c2b5b045a0b1a94"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.414326 14892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:06.421391 14892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:06.423175 14892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:06.423283 14892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:06.427143 14892 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.189:36405
I20260810 23:03:06.427232 15025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.189:36405 every 8 connection(s)
I20260810 23:03:06.427563 14892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/data/info.pb
I20260810 23:03:06.428669 15026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:06.430217 15026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:06.436790 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 14892
I20260810 23:03:06.436911 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-1/wal/instance
I20260810 23:03:06.437131 6334 external_mini_cluster.cc:1428] Setting key a1cece9def6f0d56ac7cc4a26f129b0a
I20260810 23:03:06.439527 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.430926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37618 (local address 127.6.47.190:41369)
0810 23:03:06.431017 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.431020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.431220 (+ 200us) server_negotiation.cc:407] Connection header received
0810 23:03:06.431731 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.431733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.431748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.431811 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.432410 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.432952 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.433911 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.434116 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.434202 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.436363 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.436365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.436365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.436373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.438105 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.438507 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.438509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.438509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.438577 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.438817 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.438818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.438818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.438970 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.439047 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.439394 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.439464 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 23:03:06.439790 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.430888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41369 (local address 127.0.0.1:37618)
0810 23:03:06.431094 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.431130 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.431604 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.431882 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.431886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.432118 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.432306 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.432309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.433055 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.433057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.433808 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.433811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.434233 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.434234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.434344 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.434874 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.434882 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.436261 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.438194 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.438201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.438202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.438425 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.438649 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.438650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.438650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.438750 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.439055 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.439063 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.439247 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.439367 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.439540 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260810 23:03:06.440481 15026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.47.188@KRBTEST.COM; defaulting to no policy
I20260810 23:03:06.448632 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.440824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60534)
0810 23:03:06.440885 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.440898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.440943 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.441182 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.441183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.441342 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.441402 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.441404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.442209 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.442210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.443088 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.443091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.443637 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.443638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.443726 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.444091 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.444098 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.445504 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.447520 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.447521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.447521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.447686 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.447895 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.447896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.447896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.447965 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.448276 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.448277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.448348 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.448440 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.448551 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 23:03:06.448738 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.440863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.6.47.189:36405)
0810 23:03:06.441012 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.441016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.441039 (+ 23us) server_negotiation.cc:407] Connection header received
0810 23:03:06.441058 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.441060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.441071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.441141 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.441478 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.442139 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.443176 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.443548 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.443603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.445596 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.445606 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.445607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.445614 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.447426 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.447765 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.447768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.447769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.447833 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.448026 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.448026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.448027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.448159 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.448279 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.448484 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.448599 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260810 23:03:06.449263 15026 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:06.449697 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.449572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37010)
0810 23:03:06.449627 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.449680 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 23:03:06.449855 14931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.47.188@KRBTEST.COM" created.
I20260810 23:03:06.450331 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.450231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37018)
0810 23:03:06.450280 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.450306 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 23:03:06.450436 15026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:40289: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
Entry for principal kudu/127.6.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab.
I20260810 23:03:06.467881 6334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfP10h8/build/release/bin/kudu
/tmp/dist-test-taskfP10h8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:40289
--webserver_interface=127.6.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.47.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 23:03:06.498570 14876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } attempt: 3
I20260810 23:03:06.499680 15026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } attempt: 1
I20260810 23:03:06.500120 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.499981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37030)
0810 23:03:06.500039 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.500081 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 23:03:06.500562 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.500333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37034)
0810 23:03:06.500484 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.500529 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
W20260810 23:03:06.500699 15026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:40289: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:06.506260 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.498964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60540)
0810 23:03:06.499039 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.499052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.499128 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.499298 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.499299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.499463 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.499536 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.499538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.500480 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.500482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.501382 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.501386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.501767 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.501768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.501871 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.502288 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.502294 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.503602 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.505343 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.505344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.505345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.505510 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.505715 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.505716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.505716 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.505770 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.506025 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.506025 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.506063 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.506128 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.506199 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 23:03:06.506371 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.499081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.6.47.189:36405)
0810 23:03:06.499166 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.499168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.499178 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:03:06.499207 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.499208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.499224 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.499288 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.499645 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.500387 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.501474 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.501684 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.501743 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.503688 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.503690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.503690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.503698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.505258 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.505587 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.505588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.505588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.505648 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.505834 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.505835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.505835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.505960 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.506031 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.506156 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.506249 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 23:03:06.506742 14876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:06.507113 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.506967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37046)
0810 23:03:06.507035 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.507078 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 23:03:06.507550 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.507427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37048)
0810 23:03:06.507486 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.507521 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 23:03:06.507673 14876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:40289: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
I20260810 23:03:06.538954 14876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } attempt: 1
I20260810 23:03:06.539494 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.539351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37064)
0810 23:03:06.539424 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.539465 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 23:03:06.539929 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.539822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37068)
0810 23:03:06.539869 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.539893 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 23:03:06.540058 14876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:40289: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
W20260810 23:03:06.553778 15040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 23:03:06.553975 15040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 23:03:06.553992 15040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 23:03:06.554047 15040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 23:03:06.555150 15040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 23:03:06.555190 15040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 23:03:06.555204 15040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 23:03:06.555261 15040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 23:03:06.555284 15040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 23:03:06.556799 15040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.47.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.47.190:41369,127.6.47.189:36405,127.6.47.188:40289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/krb5kdc/kudu_127.6.47.188.keytab
--principal=kudu/127.6.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.47.188:40289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 10 Aug 2026 22:08:11 UTC on ccycloud.quasar-zbnufz.root.comops.site
build id 4579241
I20260810 23:03:06.557044 15040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 23:03:06.557289 15040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 23:03:06.562377 15055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:06.562400 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 23:03:06.562402 15052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 23:03:06.562456 15040 server_base.cc:1034] running on GCE node
I20260810 23:03:06.562851 15040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 23:03:06.563190 15040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 23:03:06.564347 15040 hybrid_clock.cc:648] HybridClock initialized: now 1786402986564329 us; error 37 us; skew 500 ppm
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 23:03:06.566794 15040 init.cc:377] Logged in from keytab as kudu/127.6.47.188@KRBTEST.COM (short username kudu)
I20260810 23:03:06.567289 15040 webserver.cc:460] Webserver started at http://127.6.47.188:46377/ using document root <none> and password file <none>
I20260810 23:03:06.567541 15040 fs_manager.cc:359] Metadata directory not provided
I20260810 23:03:06.567588 15040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 23:03:06.567770 15040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 23:03:06.568826 15026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } attempt: 2
I20260810 23:03:06.568933 15040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data/instance:
uuid: "cd5268e90d35498ba36ef3074b0d7b07"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "b31510b6748b837c14fcd281acfed7a4"
server_key_iv: "460558f9a36b414160008dd5a1b468fc"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.569278 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.569087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37082)
0810 23:03:06.569158 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.569192 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:03:06.569505 15040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal/instance:
uuid: "cd5268e90d35498ba36ef3074b0d7b07"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "b31510b6748b837c14fcd281acfed7a4"
server_key_iv: "460558f9a36b414160008dd5a1b468fc"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.569708 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.569571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37096)
0810 23:03:06.569630 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.569680 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 23:03:06.569839 15026 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.47.188:40289: Network error: Client connection negotiation failed: client connection to 127.6.47.188:40289: connect: Connection refused (error 111)
I20260810 23:03:06.570940 15040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 23:03:06.571631 15072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 23:03:06.571835 15040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 23:03:06.571910 15040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data,/tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal
uuid: "cd5268e90d35498ba36ef3074b0d7b07"
format_stamp: "Formatted at 2026-08-10 23:03:06 on dist-test-slave-rwrg"
server_key: "b31510b6748b837c14fcd281acfed7a4"
server_key_iv: "460558f9a36b414160008dd5a1b468fc"
server_key_version: "encryptionkey@0"
I20260810 23:03:06.571955 15040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 23:03:06.585148 15040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 23:03:06.587410 15040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 23:03:06.587520 15040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 23:03:06.591807 15040 rpc_server.cc:307] RPC server started. Bound to: 127.6.47.188:40289
I20260810 23:03:06.591889 15171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.47.188:40289 every 8 connection(s)
I20260810 23:03:06.592196 15040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/data/info.pb
I20260810 23:03:06.593228 6334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfP10h8/build/release/bin/kudu as pid 15040
I20260810 23:03:06.593340 6334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfP10h8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786402701710467-6334-0/minicluster-data/master-2/wal/instance
I20260810 23:03:06.593389 15174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 23:03:06.593567 6334 external_mini_cluster.cc:1428] Setting key 993f3a9c5ea1a9563ed6f8ab86d4fd8e
I20260810 23:03:06.595724 15174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:06.604224 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.594100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.6.47.188:40289)
0810 23:03:06.594297 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.594306 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.594339 (+ 33us) server_negotiation.cc:407] Connection header received
0810 23:03:06.594832 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.594839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.594882 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.594989 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.595782 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.596941 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.597769 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.598231 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.598302 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.600851 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.600860 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.600860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.600868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.602683 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603086 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.603089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.603089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.603198 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603428 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.603429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.603429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.603651 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.603816 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.603927 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.604051 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260810 23:03:06.604327 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.596368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.190:41369 (local address 127.0.0.1:37632)
0810 23:03:06.596543 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.596580 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.596657 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.596810 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.596812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.597132 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.597216 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.597219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.597876 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.597877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.598579 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.598582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.598963 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.598964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.599027 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.599483 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.599491 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.601306 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.603052 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.603062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.603292 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.603536 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603536 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.603537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.603605 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.603882 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.603890 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.603941 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.604050 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.604182 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":17,"thread_start_us":107,"threads_started":1}
I20260810 23:03:06.604339 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.596450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.6.47.190:41369)
0810 23:03:06.596516 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.596518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.596666 (+ 148us) server_negotiation.cc:407] Connection header received
0810 23:03:06.596697 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.596698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.596708 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.596768 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.597312 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.597778 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.598670 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.598854 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.598933 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.601393 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.601394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.601395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.601402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.602940 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603396 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.603397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.603398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.603461 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.603693 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.603694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.603695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.603808 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.603889 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.604173 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.604250 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 23:03:06.605064 15174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.190@KRBTEST.COM
I20260810 23:03:06.615221 14876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } attempt: 2
I20260810 23:03:06.615271 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.605359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.189:36405 (local address 127.0.0.1:60554)
0810 23:03:06.605426 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.605438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.605474 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.605646 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.605647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.605763 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.605792 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.605793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.606530 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.606531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.607084 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.607086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.607567 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.607568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.607623 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.611280 (+ 3657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.611284 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.612538 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.614150 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.614151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.614151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.614326 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.614545 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.614545 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.614546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.614607 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.614906 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.614907 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.614964 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.615063 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.615169 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 23:03:06.615335 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.605450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.6.47.189:36405)
0810 23:03:06.605519 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.605521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.605529 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:03:06.605554 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.605555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.605566 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.605623 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.605877 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.606452 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.607186 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.607495 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.607551 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.612678 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.612680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.612682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.612690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.614057 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.614407 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.614409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.614409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.614466 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.614689 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.614690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.614691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.614826 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.614899 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.615109 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.615212 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 23:03:06.615909 15174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } has no permanent_uuid. Determining permanent_uuid...
I20260810 23:03:06.616199 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.605202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.6.47.190:41369)
0810 23:03:06.605417 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.605419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.605426 (+ 7us) server_negotiation.cc:407] Connection header received
0810 23:03:06.605445 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.605445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.605454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.605538 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.606157 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.606647 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.607409 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.607643 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.607714 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.613152 (+ 5438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.613154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.613154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.613161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.614667 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.615027 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.615028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.615028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.615090 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.615343 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.615345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.615345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.615480 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.615631 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.615863 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.616112 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.190@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.188@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.47.189@KRBTEST.COM
I20260810 23:03:06.624771 15184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.616152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37124)
0810 23:03:06.616215 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.616227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.616267 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.616479 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.616479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.616682 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.616709 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.616711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.618096 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.618097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.619175 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.619178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.619613 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.619614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.619663 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.620034 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.620038 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.621405 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.623756 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.623757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.623757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.623924 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.624137 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.624138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.624138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.624190 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.624438 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.624439 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.624471 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.624539 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.624630 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 23:03:06.624806 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.615467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37118)
0810 23:03:06.615511 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.615523 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.615559 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.615736 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.615737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.615950 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.615991 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.615993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.617161 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.617162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.617770 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.617773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.618298 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.618299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.618361 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.618802 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.618807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.621941 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.623574 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.623575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.623575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.623809 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.624049 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.624050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.624050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.624110 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.624408 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.624409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.624452 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.624594 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.624710 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260810 23:03:06.624835 15183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.615564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.6.47.188:40289)
0810 23:03:06.615598 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.615600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.615624 (+ 24us) server_negotiation.cc:407] Connection header received
0810 23:03:06.615643 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.615644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.615652 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.615942 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.616342 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.617074 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.617865 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.618212 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.618278 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.622044 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.622045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.622045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.622052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.623485 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.623909 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.623910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.623911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.623970 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.624187 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.624188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.624188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.624324 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.624402 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.624624 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.624704 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:03:06.625144 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.616189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.6.47.188:40289)
0810 23:03:06.616368 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.616369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.616375 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:03:06.616391 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.616392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.616401 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.616761 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.616818 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.618001 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.619269 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.619531 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.619614 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.621497 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.621498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.621498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.621505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.623642 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.624003 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.624004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.624004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.624062 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.624255 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.624256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.624256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.624371 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.624891 (+ 520us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.624932 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.625010 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":115,"threads_started":1}
I20260810 23:03:06.626175 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.616852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60564 (local address 127.6.47.189:36405)
0810 23:03:06.616885 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.616886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.616910 (+ 24us) server_negotiation.cc:407] Connection header received
0810 23:03:06.616928 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.616929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.616937 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.616979 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.617647 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.618339 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.619018 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.619317 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.619381 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.622092 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.622094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.622094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.622102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.624771 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.625344 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.625346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.625346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.625415 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.625642 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.625643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.625644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.625764 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.625967 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.626004 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.626073 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260810 23:03:06.626713 15174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: Bootstrap starting.
I20260810 23:03:06.627010 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: Bootstrap starting.
I20260810 23:03:06.627457 15174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:06.627697 14876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:06.627775 15174 log.cc:824] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:06.627986 14876 log.cc:824] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:06.628659 15174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: No bootstrap required, opened a new log
I20260810 23:03:06.628952 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: No bootstrap required, opened a new log
I20260810 23:03:06.629650 6334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 23:03:06.630628 14832 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7602705a2d3f4ab8b089a5b2458a01d8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37640
I20260810 23:03:06.630735 15174 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.630892 15174 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5268e90d35498ba36ef3074b0d7b07, State: Initialized, Role: FOLLOWER
I20260810 23:03:06.630873 14876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.631003 14876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7602705a2d3f4ab8b089a5b2458a01d8, State: Initialized, Role: FOLLOWER
I20260810 23:03:06.630987 15174 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.631116 14876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.631337 15174 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:06.631409 14876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:06.631470 14832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 23:03:06.631521 14832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:06.632470 14832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.632791 14832 leader_election.cc:288] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [CANDIDATE]: Term 1 election: Requested vote from peers 5d2a786f26674b5db91273eb60fafb7a (127.6.47.189:36405), cd5268e90d35498ba36ef3074b0d7b07 (127.6.47.188:40289)
I20260810 23:03:06.633466 15126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd5268e90d35498ba36ef3074b0d7b07"
I20260810 23:03:06.633594 15126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [term 0 FOLLOWER]: Advancing to term 1
W20260810 23:03:06.633795 15227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:03:06.633846 15227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:03:06.634459 15126 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7602705a2d3f4ab8b089a5b2458a01d8 in term 1.
I20260810 23:03:06.634801 14779 leader_election.cc:302] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7602705a2d3f4ab8b089a5b2458a01d8, cd5268e90d35498ba36ef3074b0d7b07; no voters:
I20260810 23:03:06.632990 15192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.634959 15192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.635061 15192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 23:03:06.635170 15192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [term 1 LEADER]: Becoming Leader. State: Replica: 7602705a2d3f4ab8b089a5b2458a01d8, State: Running, Role: LEADER
I20260810 23:03:06.631357 15190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.635274 15192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.635226 15190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 23:03:06.636373 15237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:03:06.636436 15237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:03:06.636776 15250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7602705a2d3f4ab8b089a5b2458a01d8. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.636870 15250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:06.637054 15253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 23:03:06.637320 15253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 23:03:06.636528 14743 master_runner.cc:186] Error getting master registration for 127.6.47.189:36405: OK, instance_id { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" instance_seqno: 1786402986426062 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:03:06.639096 14743 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:03:06.639933 15040 master_runner.cc:186] Error getting master registration for 127.6.47.189:36405: OK, instance_id { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" instance_seqno: 1786402986426062 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 23:03:06.643134 15040 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 23:03:06.647353 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.635778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.6.47.190:41369)
0810 23:03:06.635836 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.635838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.637995 (+ 2157us) server_negotiation.cc:407] Connection header received
0810 23:03:06.638097 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.638098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.638108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.638138 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.639020 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.639751 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.641588 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.641887 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.641929 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.643844 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.643845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.643846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.643853 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.646074 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.646531 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.646532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.646532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.646618 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.646821 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.646822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.646822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.646956 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.647077 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.647183 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.647259 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 23:03:06.649715 15126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [term 1 FOLLOWER]: Refusing update from remote peer 7602705a2d3f4ab8b089a5b2458a01d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:03:06.650228 15250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:03:06.651633 15190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7602705a2d3f4ab8b089a5b2458a01d8. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.651753 15190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.653327 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.637197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.6.47.188:40289)
0810 23:03:06.637233 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.637234 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.641719 (+ 4485us) server_negotiation.cc:407] Connection header received
0810 23:03:06.641814 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.641815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.641827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.641855 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.643019 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.643699 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.644889 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.645200 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.645292 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.648602 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.648604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.648604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.648613 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.650591 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.651830 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.651831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.651832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.651882 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.652401 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.652403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.652403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.652572 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.652921 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.653140 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.653233 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:03:06.654954 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.637700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.6.47.189:36405)
0810 23:03:06.637750 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.637752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.641663 (+ 3911us) server_negotiation.cc:407] Connection header received
0810 23:03:06.642769 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.642771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.642781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.642809 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.644944 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.645636 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.649372 (+ 3736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.649701 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.649747 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.651796 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.651797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.651798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.651807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.653615 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.654115 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.654116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.654117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.654196 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.654423 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.654424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.654424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.654554 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.654665 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.654787 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.654867 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260810 23:03:06.661134 15026 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } attempt: 3
I20260810 23:03:06.662444 15250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.662542 15250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:06.663012 15190 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.663096 15190 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.663242 15263 mvcc.cc:205] Tried to move back new op lower bound from 7317106633311326208 to 7317106633258930176. Current Snapshot: MvccSnapshot[applied={T|T < 7317106633311326208}]
I20260810 23:03:06.663340 15264 mvcc.cc:205] Tried to move back new op lower bound from 7317106633311326208 to 7317106633258930176. Current Snapshot: MvccSnapshot[applied={T|T < 7317106633311326208}]
I20260810 23:03:06.664229 15253 catalog_manager.cc:1347] Generated new cluster ID: 801b870b8cda432898cc1dc61a1994f2
I20260810 23:03:06.664644 15253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 10 23:03:06 dist-test-slave-rwrg krb5kdc[14720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786402986, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.47.189@KRBTEST.COM for kudu/127.6.47.188@KRBTEST.COM
I20260810 23:03:06.669324 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.661383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.47.188:40289 (local address 127.0.0.1:37140)
0810 23:03:06.661432 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0810 23:03:06.661444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 23:03:06.661482 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 23:03:06.661644 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 23:03:06.661645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 23:03:06.661815 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0810 23:03:06.661876 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.661878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.662763 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.662764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.663593 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 23:03:06.663597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.664040 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.664041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 23:03:06.664099 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.664623 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 23:03:06.664628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 23:03:06.666379 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 23:03:06.668173 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.668174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.668175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.668399 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.668607 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 23:03:06.668608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 23:03:06.668608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0810 23:03:06.668673 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 23:03:06.668969 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 23:03:06.668970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 23:03:06.669016 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 23:03:06.669094 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 23:03:06.669198 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 23:03:06.669324 15186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:06.661419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.6.47.188:40289)
0810 23:03:06.661457 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.661459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.661469 (+ 10us) server_negotiation.cc:407] Connection header received
0810 23:03:06.661541 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.661542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.661552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.661598 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.661954 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.662689 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.663670 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:06.663978 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:06.664043 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:06.666453 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 23:03:06.666455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 23:03:06.666455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 23:03:06.666462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0810 23:03:06.668086 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.668471 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.668472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.668472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.668550 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 23:03:06.668745 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 23:03:06.668746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 23:03:06.668746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0810 23:03:06.668877 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 23:03:06.668962 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:06.669128 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 23:03:06.669199 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:06.670922 15026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: Bootstrap starting.
I20260810 23:03:06.671741 15026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: Neither blocks nor log segments found. Creating new log.
I20260810 23:03:06.672053 15026 log.cc:824] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: Log is configured to *not* fsync() on all Append() calls
I20260810 23:03:06.673470 15026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: No bootstrap required, opened a new log
I20260810 23:03:06.675661 15026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.675807 15026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d2a786f26674b5db91273eb60fafb7a, State: Initialized, Role: FOLLOWER
I20260810 23:03:06.675895 15026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } }
I20260810 23:03:06.676291 15026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 23:03:06.676321 14974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5d2a786f26674b5db91273eb60fafb7a"
I20260810 23:03:06.676397 14974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [term 0 FOLLOWER]: Advancing to term 1
I20260810 23:03:06.677404 14974 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7602705a2d3f4ab8b089a5b2458a01d8 in term 1.
I20260810 23:03:06.677626 14972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [term 1 FOLLOWER]: Refusing update from remote peer 7602705a2d3f4ab8b089a5b2458a01d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 23:03:06.676213 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.677994 15192 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 23:03:06.678037 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.679335 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7602705a2d3f4ab8b089a5b2458a01d8. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.679468 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.679543 15274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.679643 15274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 23:03:06.681191 15295 mvcc.cc:205] Tried to move back new op lower bound from 7317106633311326208 to 7317106633258930176. Current Snapshot: MvccSnapshot[applied={T|T < 7317106633311326208}]
W20260810 23:03:06.685089 15298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 23:03:06.685192 15298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 23:03:06.695114 15192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7602705a2d3f4ab8b089a5b2458a01d8" member_type: VOTER last_known_addr { host: "127.6.47.190" port: 41369 } } peers { permanent_uuid: "5d2a786f26674b5db91273eb60fafb7a" member_type: VOTER last_known_addr { host: "127.6.47.189" port: 36405 } } peers { permanent_uuid: "cd5268e90d35498ba36ef3074b0d7b07" member_type: VOTER last_known_addr { host: "127.6.47.188" port: 40289 } } }
I20260810 23:03:06.695228 15192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8 [sys.catalog]: This master's current role is: LEADER
I20260810 23:03:06.696969 15253 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 23:03:06.697700 15253 catalog_manager.cc:1504] Loading token signing keys...
I20260810 23:03:06.703168 15253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7602705a2d3f4ab8b089a5b2458a01d8: Generated new TSK 0
I20260810 23:03:06.703341 15253 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 23:03:06.731566 15186 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.729773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.6.47.188:40289)
0810 23:03:06.729820 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.729823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.730834 (+ 1011us) server_negotiation.cc:407] Connection header received
0810 23:03:06.730948 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.730950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.730962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.731015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.731497 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37142: BlockingRecv error: recv got EOF from 127.0.0.1:37142 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260810 23:03:06.732585 14880 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.730182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.6.47.190:41369)
0810 23:03:06.730224 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.730226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.731827 (+ 1601us) server_negotiation.cc:407] Connection header received
0810 23:03:06.732052 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.732054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.732066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.732116 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.732479 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37646: BlockingRecv error: recv got EOF from 127.0.0.1:37646 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:03:06.733453 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.730486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.6.47.189:36405)
0810 23:03:06.730524 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.730526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.732827 (+ 2301us) server_negotiation.cc:407] Connection header received
0810 23:03:06.732931 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.732932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.732944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.732999 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.733387 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60576: BlockingRecv error: recv got EOF from 127.0.0.1:60576 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260810 23:03:06.735882 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.735063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.6.47.189:36405)
0810 23:03:06.735098 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.735100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.735354 (+ 254us) server_negotiation.cc:407] Connection header received
0810 23:03:06.735459 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.735460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.735478 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.735507 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.735839 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60590: BlockingRecv error: recv got EOF from 127.0.0.1:60590 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:03:06.738875 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.738126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.6.47.189:36405)
0810 23:03:06.738159 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.738161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.738371 (+ 210us) server_negotiation.cc:407] Connection header received
0810 23:03:06.738468 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.738468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.738479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.738508 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.738833 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60602: BlockingRecv error: recv got EOF from 127.0.0.1:60602 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260810 23:03:06.742789 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.741991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.6.47.189:36405)
0810 23:03:06.742024 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.742026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.742247 (+ 221us) server_negotiation.cc:407] Connection header received
0810 23:03:06.742371 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.742372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.742383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.742411 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.742748 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60610: BlockingRecv error: recv got EOF from 127.0.0.1:60610 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260810 23:03:06.748780 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.747946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.6.47.189:36405)
0810 23:03:06.747981 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.747982 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.748210 (+ 228us) server_negotiation.cc:407] Connection header received
0810 23:03:06.748310 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.748311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.748321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.748350 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.748736 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60618: BlockingRecv error: recv got EOF from 127.0.0.1:60618 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 23:03:06.758905 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.758042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.6.47.189:36405)
0810 23:03:06.758079 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.758081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.758315 (+ 234us) server_negotiation.cc:407] Connection header received
0810 23:03:06.758420 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.758421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.758433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.758463 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.758853 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60624: BlockingRecv error: recv got EOF from 127.0.0.1:60624 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260810 23:03:06.777401 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.776361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.6.47.189:36405)
0810 23:03:06.776410 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.776413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.776774 (+ 361us) server_negotiation.cc:407] Connection header received
0810 23:03:06.776890 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.776891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.776905 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.776938 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.777336 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60634: BlockingRecv error: recv got EOF from 127.0.0.1:60634 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 23:03:06.811704 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.810856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60644 (local address 127.6.47.189:36405)
0810 23:03:06.810906 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.810908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.811076 (+ 168us) server_negotiation.cc:407] Connection header received
0810 23:03:06.811163 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.811164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.811187 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.811260 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.811631 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60644: BlockingRecv error: recv got EOF from 127.0.0.1:60644 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260810 23:03:06.877624 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:06.876808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.6.47.189:36405)
0810 23:03:06.876884 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:06.876886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:06.877055 (+ 169us) server_negotiation.cc:407] Connection header received
0810 23:03:06.877142 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:06.877143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:06.877156 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:06.877199 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:06.877557 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60656: BlockingRecv error: recv got EOF from 127.0.0.1:60656 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260810 23:03:07.007687 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:07.006904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.6.47.189:36405)
0810 23:03:07.006970 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.006972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.007080 (+ 108us) server_negotiation.cc:407] Connection header received
0810 23:03:07.007137 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.007138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.007148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.007215 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:07.007578 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60658: BlockingRecv error: recv got EOF from 127.0.0.1:60658 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 23:03:07.265762 15034 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 23:03:07.265073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.6.47.189:36405)
0810 23:03:07.265157 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.265163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.265174 (+ 11us) server_negotiation.cc:407] Connection header received
0810 23:03:07.265201 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.265202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.265214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.265268 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 23:03:07.265662 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60662: BlockingRecv error: recv got EOF from 127.0.0.1:60662 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 23:03:07.634286 15227 catalog_manager.cc:1259] Loaded cluster ID: 801b870b8cda432898cc1dc61a1994f2
I20260810 23:03:07.634366 15227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: loading cluster ID for follower catalog manager: success
I20260810 23:03:07.635121 15227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: acquiring CA information for follower catalog manager: success
I20260810 23:03:07.635368 15227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd5268e90d35498ba36ef3074b0d7b07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:03:07.685748 15298 catalog_manager.cc:1259] Loaded cluster ID: 801b870b8cda432898cc1dc61a1994f2
I20260810 23:03:07.685844 15298 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: loading cluster ID for follower catalog manager: success
I20260810 23:03:07.686661 15298 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: acquiring CA information for follower catalog manager: success
I20260810 23:03:07.686949 15298 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d2a786f26674b5db91273eb60fafb7a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 23:03:07.781540 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:07.779009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.6.47.189:36405)
0810 23:03:07.779142 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.779144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.779156 (+ 12us) server_negotiation.cc:407] Connection header received
0810 23:03:07.779224 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.779225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.779278 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.779343 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:07.779494 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.779991 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.780805 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.780995 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.781054 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:07.781228 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:07.781335 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:07.781415 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:07.781455 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 23:03:07.781473 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260810 23:03:07.786053 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:07.783409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.6.47.188:40289)
0810 23:03:07.783550 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.783552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.783560 (+ 8us) server_negotiation.cc:407] Connection header received
0810 23:03:07.783605 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.783606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.783655 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.783750 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:07.783876 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.784368 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.785232 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.785448 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.785530 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:07.785702 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:07.785813 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:07.785902 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:07.785941 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 23:03:07.785963 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
I20260810 23:03:07.790882 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:07.787931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.6.47.189:36405)
0810 23:03:07.787968 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.787970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.788200 (+ 230us) server_negotiation.cc:407] Connection header received
0810 23:03:07.788276 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.788277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.788287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.788340 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:07.788473 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.788990 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.790092 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.790388 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.790449 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:07.790611 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:07.790650 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:07.790725 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:07.790789 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 23:03:07.790809 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260810 23:03:07.791136 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:07.787758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.6.47.190:41369)
0810 23:03:07.788153 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.788155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.788161 (+ 6us) server_negotiation.cc:407] Connection header received
0810 23:03:07.788226 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.788227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.788279 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.788393 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:07.788485 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.789201 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.789983 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.790280 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.790348 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:07.790552 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:07.790654 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:07.790994 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:07.791014 (+ 20us) server_negotiation.cc:299] Negotiation successful
0810 23:03:07.791042 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":58,"threads_started":1}
I20260810 23:03:07.791316 15455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 23:03:07.787818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37168 (local address 127.6.47.188:40289)
0810 23:03:07.787857 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0810 23:03:07.787860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 23:03:07.787889 (+ 29us) server_negotiation.cc:407] Connection header received
0810 23:03:07.787978 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 23:03:07.787979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 23:03:07.787988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 23:03:07.788049 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 23:03:07.788218 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.788958 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.790509 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 23:03:07.790790 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 23:03:07.790850 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 23:03:07.791051 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 23:03:07.791080 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 23:03:07.791122 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 23:03:07.791234 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 23:03:07.791255 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260810 23:03:07.792927 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14743
I20260810 23:03:07.808571 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 14892
I20260810 23:03:07.822068 6334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfP10h8/build/release/bin/kudu with pid 15040
2026-08-10T23:03:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3099 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6166 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (286131 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST